Too much string.Concat in generated code with large blocks of HTML

Take a gander at the generated code for a page with a lot of 'literal text'
https://github.com/rynowak/mvc-sandbox/blob/master/workloads/RazorCodeGenerator/LargePage.cshtml
namespace Test
{
    using System;
    using System.Linq;
    using System.Collections.Generic;
    using Microsoft.AspNet.Mvc;
    using Microsoft.AspNet.Mvc.Rendering;
    using System.Threading.Tasks;

    public class LargePage : Microsoft.AspNet.Mvc.Razor.RazorPage<dynamic>
    {
        #line hidden
        public LargePage()
        {
        }
        #line hidden
        [Microsoft.AspNet.Mvc.Razor.Internal.RazorInjectAttribute]
        public Microsoft.AspNet.Mvc.IUrlHelper Url { get; private set; }
        [Microsoft.AspNet.Mvc.Razor.Internal.RazorInjectAttribute]
        public Microsoft.AspNet.Mvc.IViewComponentHelper Component { get; private set; }
        [Microsoft.AspNet.Mvc.Razor.Internal.RazorInjectAttribute]
        public Microsoft.AspNet.Mvc.Rendering.IJsonHelper Json { get; private set; }
        [Microsoft.AspNet.Mvc.Razor.Internal.RazorInjectAttribute]
        public Microsoft.AspNet.Mvc.Rendering.IHtmlHelper<dynamic> Html { get; private set; }

        #line hidden

        #pragma warning disable 1998
        public override async Task ExecuteAsync()
        {
            BeginContext(0, 160056, true);
            WriteLiteral("
Paragraph

Paragraph

Paragraph

Paragraph

Paragraph
<" +
"p>Paragraph

Paragraph

Paragraph

Paragraph

Paragraph
<" +
"p>Paragraph

Paragraph

Paragraph

Paragraph

Paragraph
<" +
"p>Paragraph

Paragraph

Paragraph

Paragraph

Paragraph
<" +
"p>Paragraph

Paragraph

Paragraph

Paragraph

Paragraph
<" +
"p>Paragraph

Paragraph

Paragraph

Paragraph

Paragraph
<" +
"p>Paragraph

Paragraph

Paragraph

Paragraph

Paragraph
<" +
"p>Paragraph

Paragraph

Paragraph

Paragraph

Paragraph
<" +
"p>Paragraph

Paragraph

Paragraph

Paragraph

Paragraph
<" +
"p>Paragraph

Paragraph

Paragraph

Paragraph

Paragraph
<" +
"p>Paragraph

Paragraph

Paragraph

Paragraph

Paragraph
<" +
"p>Paragraph

Paragraph

Paragraph

Paragraph

Paragraph
<" +
"p>Paragraph

Paragraph

Paragraph

Paragraph

Paragraph
<" +
"p>Paragraph

Paragraph

Paragraph

Paragraph

Paragraph
<" +
"p>Paragraph

Paragraph

Paragraph

Paragraph

Paragraph
<" +
"p>Paragraph

Paragraph

Paragraph

Paragraph

Paragraph
<" +
"p>Paragraph

Paragraph

Paragraph

Paragraph

Paragraph
<" +
"p>Paragraph

Paragraph

Paragraph

Paragraph

Paragraph
<" +
...

Yeah, that's a string.Concat that's going to make a 160056 character string.
We need to make Razor smarter.

I'm surprised that this is using string.Concat() at all. Why does the compiler not fold these into one large string literal? Is this about debug/release? An optimization that doesn't exist in Roslyn perhaps?

For example, this code:
            var x = "a" + "b";
            Console.WriteLine(x);

Compiled into this (via ILSpy):
	string value = "ab";
	Console.WriteLine(value);

And similar code is in the IL:
	IL_0008: ldstr "ab"

I tried with strings that add up to ~300KB and I got the same results each time: all the string literals were folded (feld? felled? felded? follied?) into one giant string.
The form of my strings was the same as in the example:
            var x =
                "p>Paragraph

Paragraph

Paragraph

Paragraph

Paragraph
<" +
                "p>Paragraph

Paragraph

Paragraph

Paragraph

Paragraph
<" +
    ...

Per @NTaylorMullen - this semantic was copied from CodeDOM, and was intended to avoid an LOH allocation. It doesn't sound like that's doing what it's supposed to do. We might need to measure more to see what's better for GC perf here - allowing the LOH string vs breaking it up into separate writes.
Ok with pushing this item out until we're in a better position to test this.

My only point was there there's no string.Concat at all in the code shown. Just one ginormous string (which has to end up somewhere in random access RAM memory).

Yeah, you're totes bringing the pain. We should still look at this though.

@Eilon - this causes an OOM sometimes during runtime compilation:
>	mscorlib.dll!string.Concat(string str0, string str1)	Unknown
 	Microsoft.CodeAnalysis.CSharp.dll!Microsoft.CodeAnalysis.CSharp.Binder.FoldStringConcatenation(Microsoft.CodeAnalysis.CSharp.BinaryOperatorKind kind, Microsoft.CodeAnalysis.ConstantValue valueLeft, Microsoft.CodeAnalysis.ConstantValue valueRight, ref int compoundStringLength)	Unknown
 	Microsoft.CodeAnalysis.CSharp.dll!Microsoft.CodeAnalysis.CSharp.Binder.FoldBinaryOperator(Microsoft.CodeAnalysis.CSharp.CSharpSyntaxNode syntax, Microsoft.CodeAnalysis.CSharp.BinaryOperatorKind kind, Microsoft.CodeAnalysis.CSharp.BoundExpression left, Microsoft.CodeAnalysis.CSharp.BoundExpression right, Microsoft.CodeAnalysis.SpecialType resultType, Microsoft.CodeAnalysis.DiagnosticBag diagnostics, ref int compoundStringLength)	Unknown
 	Microsoft.CodeAnalysis.CSharp.dll!Microsoft.CodeAnalysis.CSharp.Binder.BindSimpleBinaryOperator(Microsoft.CodeAnalysis.CSharp.Syntax.BinaryExpressionSyntax node, Microsoft.CodeAnalysis.DiagnosticBag diagnostics, Microsoft.CodeAnalysis.CSharp.BoundExpression left, Microsoft.CodeAnalysis.CSharp.BoundExpression right, ref int compoundStringLength)	Unknown
 	Microsoft.CodeAnalysis.CSharp.dll!Microsoft.CodeAnalysis.CSharp.Binder.BindSimpleBinaryOperator(Microsoft.CodeAnalysis.CSharp.Syntax.BinaryExpressionSyntax node, Microsoft.CodeAnalysis.DiagnosticBag diagnostics)	Unknown
 	Microsoft.CodeAnalysis.CSharp.dll!Microsoft.CodeAnalysis.CSharp.Binder.BindExpressionInternal(Microsoft.CodeAnalysis.CSharp.Syntax.ExpressionSyntax node, Microsoft.CodeAnalysis.DiagnosticBag diagnostics, bool invoked, bool indexed)	Unknown
 	Microsoft.CodeAnalysis.CSharp.dll!Microsoft.CodeAnalysis.CSharp.Binder.BindArgumentAndName(Microsoft.CodeAnalysis.CSharp.AnalyzedArguments result, Microsoft.CodeAnalysis.DiagnosticBag diagnostics, bool hadError, Microsoft.CodeAnalysis.CSharp.CSharpSyntaxNode argumentSyntax, Microsoft.CodeAnalysis.CSharp.Syntax.ExpressionSyntax argumentExpression, Microsoft.CodeAnalysis.CSharp.Syntax.NameColonSyntax nameColonSyntax, Microsoft.CodeAnalysis.RefKind refKind, bool allowArglist)	Unknown
 	Microsoft.CodeAnalysis.CSharp.dll!Microsoft.CodeAnalysis.CSharp.Binder.BindArgumentAndName(Microsoft.CodeAnalysis.CSharp.AnalyzedArguments result, Microsoft.CodeAnalysis.DiagnosticBag diagnostics, bool hadError, Microsoft.CodeAnalysis.CSharp.Syntax.ArgumentSyntax argumentSyntax, bool allowArglist, bool isDelegateCreation)	Unknown
 	Microsoft.CodeAnalysis.CSharp.dll!Microsoft.CodeAnalysis.CSharp.Binder.BindInvocationExpression(Microsoft.CodeAnalysis.CSharp.Syntax.InvocationExpressionSyntax node, Microsoft.CodeAnalysis.DiagnosticBag diagnostics)	Unknown
 	Microsoft.CodeAnalysis.CSharp.dll!Microsoft.CodeAnalysis.CSharp.Binder.BindExpressionInternal(Microsoft.CodeAnalysis.CSharp.Syntax.ExpressionSyntax node, Microsoft.CodeAnalysis.DiagnosticBag diagnostics, bool invoked, bool indexed)	Unknown
 	Microsoft.CodeAnalysis.CSharp.dll!Microsoft.CodeAnalysis.CSharp.Binder.BindExpression(Microsoft.CodeAnalysis.CSharp.Syntax.ExpressionSyntax node, Microsoft.CodeAnalysis.DiagnosticBag diagnostics, bool invoked, bool indexed)	Unknown
 	Microsoft.CodeAnalysis.CSharp.dll!Microsoft.CodeAnalysis.CSharp.Binder.BindExpressionStatement(Microsoft.CodeAnalysis.CSharp.CSharpSyntaxNode node, Microsoft.CodeAnalysis.CSharp.Syntax.ExpressionSyntax syntax, bool allowsAnyExpression, Microsoft.CodeAnalysis.DiagnosticBag diagnostics)	Unknown
 	Microsoft.CodeAnalysis.CSharp.dll!Microsoft.CodeAnalysis.CSharp.Binder.BindStatement(Microsoft.CodeAnalysis.CSharp.Syntax.StatementSyntax node, Microsoft.CodeAnalysis.DiagnosticBag diagnostics)	Unknown
 	Microsoft.CodeAnalysis.CSharp.dll!Microsoft.CodeAnalysis.CSharp.Binder.BindBlock(Microsoft.CodeAnalysis.CSharp.Syntax.BlockSyntax node, Microsoft.CodeAnalysis.DiagnosticBag diagnostics, Microsoft.CodeAnalysis.CSharp.Binder blockBinder)	Unknown
 	Microsoft.CodeAnalysis.CSharp.dll!Microsoft.CodeAnalysis.CSharp.MethodCompiler.BindMethodBody(Microsoft.CodeAnalysis.CSharp.Symbols.MethodSymbol method, Microsoft.CodeAnalysis.CSharp.TypeCompilationState compilationState, Microsoft.CodeAnalysis.DiagnosticBag diagnostics, out Microsoft.CodeAnalysis.CSharp.ImportChain importChain)	Unknown
 	Microsoft.CodeAnalysis.CSharp.dll!Microsoft.CodeAnalysis.CSharp.MethodCompiler.CompileMethod(Microsoft.CodeAnalysis.CSharp.Symbols.MethodSymbol methodSymbol, int methodOrdinal, ref Microsoft.CodeAnalysis.CSharp.Binder.ProcessedFieldInitializers processedInitializers, Microsoft.CodeAnalysis.CSharp.SynthesizedSubmissionFields previousSubmissionFields, Microsoft.CodeAnalysis.CSharp.TypeCompilationState compilationState)	Unknown
 	Microsoft.CodeAnalysis.CSharp.dll!Microsoft.CodeAnalysis.CSharp.MethodCompiler.CompileNamedType(Microsoft.CodeAnalysis.CSharp.Symbols.NamedTypeSymbol containingType)	Unknown
 	Microsoft.CodeAnalysis.CSharp.dll!Microsoft.CodeAnalysis.CSharp.MethodCompiler.CompileNamedTypeAsTask.AnonymousMethod__0()	Unknown
 	Microsoft.CodeAnalysis.dll!Roslyn.Utilities.UICultureUtilities.WithCurrentUICulture.AnonymousMethod__0()	Unknown
 	mscorlib.dll!System.Threading.Tasks.Task.InnerInvoke()	Unknown
 	mscorlib.dll!System.Threading.Tasks.Task.Execute()	Unknown
 	mscorlib.dll!System.Threading.Tasks.Task.ExecutionContextCallback(object obj)	Unknown
 	mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)	Unknown
 	mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)	Unknown
 	mscorlib.dll!System.Threading.Tasks.Task.ExecuteWithThreadLocal(ref System.Threading.Tasks.Task currentTaskSlot)	Unknown
 	mscorlib.dll!System.Threading.Tasks.Task.ExecuteEntry(bool bPreventDoubleExecution)	Unknown
 	mscorlib.dll!System.Threading.Tasks.Task.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()	Unknown
 	mscorlib.dll!System.Threading.ThreadPoolWorkQueue.Dispatch()	Unknown
 	mscorlib.dll!System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()	Unknown

@rynowak should we also be filing a Roslyn bug? The memory quantities in play here are not really that much. A few hundred KB is a drop in the bucket even in a 32bit process.

I think we should file a Roslyn bug if we're still seeing this on dotnet cli. We already dump a ton of stuff into LOH, and it's fragmentation that's likely causing this.

Nice memory stress test :smile:

https://github.com/dotnet/roslyn/issues/7398

This is actually known and also happens when we implemented the roslyn codedom based compiler.
/cc @DamianEdwards

It's pretty obviously known since there's a bug. Are you trying to make another point? We need to fix this.

That we brought it up with the roslyn team back then and I believe the answer was to compile out of process to avoid bringing down the server with an OOM. I'm not sure there was a real solution per se.

But the OOM would happen anyway, no? I mean, something somewhere is going to get an OOM and compilation will fail. Don't care which process is crashing, my app doesn't work.
But either way we're going to need to avoid this problem in Razor so it's not as big an issue for us.

The difference can be that one view/page fails to compile vs. the entire app.

Razor needs to break up long literals every X chars and call WriteLiteral for each one. This avoid the Roslyn bug, and might even have other benefits.
I think the only question is what "X" should be. 1KB? 10KB? 50KB?

Could solve this bug by moving a special case from CSharpCodeWriter to CSharpCodeVisitor.
See CSharpCodeWriter.WriteStringLiteral() which special-cases any string shorter than 256 or longer than 1500 characters (both exclusive) or which contains a \0 character. In that special case, WriteCStyleStringLiteral() breaks the generated string into bits of at most 81 characters. Yes, the bits are joined together using "\" +\n\"".
Most WriteStringLiteral() callers deal with attributes names and values or similar strings that are highly unlikely to need such a broad special case (maybe one just for \0). Just two call sites (in CSharpCodeVisitor) seem to merit length-related special handling and both are generating calls to RazorPage.WriteLiteral() or RazorPage.WriteLiteralTo(). Do the length handling there instead.
(Side note: CSharpCodeWriter chose X to be 80 though that seems to be for style in the generated code.)

605dceea020bd41ff4e70f05be04733f57bbb75f


Custom colors for text

Link to feature: https://adaptivecards.productboard.com/feature-board/planning/features/2182571


Linter doesn't work in ST3?

I've installed this plugin in SublimeText 3 (build 3126), and when I enter the command "Toggle Linter" it doesn't show up in the list. Also, "Lint this View" doesn't do anything apparent.

Not sure, but it'll surely stop working very soon with SL4.


printEnabled set to false doesn't work

I have the following setting in Background section:
* configure printEnabled = false
as I don't want to print out responses in scenarios, but they still are being printed.
It is a bug or does the configuration should look differently?
(Karate v. 1.0.1)

I think you are looking for * configure report = { showLog: false } please read the docs: https://github.com/intuit/karate#report-verbosity
if there is still a bug (quite possible) please follow this process: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue

No @ptrthomas , I'm refering to https://github.com/intuit/karate#configure. There is this option printEnabled with following description:
"Can be used to suppress the print output when not in 'dev mode' by setting as false (default true)"
I have several "prints" in my scenarios (for debugging purpose), but when executing against other environments than dev, I want to disable prints, and I thought this is the right option.
I could follow the how-to-submit-an-issue process, but this is really simple to verify. Just put the line in Background and check if print is enabled or disabled.

@intest I'll reopen it - but tagging as help wanted. not considering this a priority. perhaps you can contribute a PR ? it should be a simple fix

Thanks @ptrthomas. Yes, it's definitelly low prio, but still...
I would love to contribute, if I had enough programming skills :)
(btw, think about donations on Patreon or similar. I can't contribute  to a code, but would donate a few bucks/month, as I found the tool really great and game-changing)
Little update about a bug:
It is working inside a scenario, for example:
When method post
Then status 200
* configure printEnabled = false
* print karate.pretty(response)

In such case print is ignored, so works as expected. The issue is that it doesn't work when applied in Background section.
PS. the same case with * configure continueOnStepFailure = true, which I'm currently testing (a feature I was looking for, so thanks btw for implementing, but would be great if it works in Background as well).

@intest yep that helps

When I was reviewing other issues, I found this one https://github.com/intuit/karate/issues/1560, and I realized that I did not mention one important thing - that the issue reported by me concerns also scenario outlines.
I tested now a regular scenario and printEnabled  (set to false in Background) works, so maybe this issue duplicates this mentioned one.

@intest it will be very helpful if you can follow the developer guide and confirm that it is fixed: https://github.com/intuit/karate/wiki/Developer-Guide

closing this since there is no response. this can be tested against 1.1.0.RC1 - if there is still an issue we need a sample

Tested against 1.1.0.RC1 and now it works. Thanks for fixing.
(but I found another minor issue, dunno if reated to this one, but I will submit it of course)


Fix git identity in Merge upstream step

Changes
Fix git identity in Merge upstream step.
For non-members of knative it seems these values are not set.
Fixes #137

/cc @dsimansk

also cc @chizhg if you have any input on how prow does this

If it has no impact on the later merged commit then LGTM

/lgtm

/approve


[Minor] fix NullPointerException

What is the purpose of the pull request
This PR fixes a NPE when compacting via hudi-cli and providing a compaction props file via propsFilePath
Brief change log

initialize FileSystem before use

Verify this pull request
This pull request is a trivial rework / code cleanup without any test coverage.
Committer checklist

[ ] Has a corresponding JIRA in PR title & commit

[x] Commit message is descriptive of the change

[ ] CI is green

[ ] Necessary doc changes done or have another open PR

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

@rolandjohann Thanks for your contributing, would you please file a jira for the PR and update the title with [HUDI-xxx].

@leesf can you please review this one?

@rolandjohann Would you please check why the travis is red?

Is it possible that this is a test infrastructure related issue?
[ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 7.416 s <<< FAILURE! - in org.apache.hudi.cli.commands.TestArchivedCommitsCommand
[ERROR] org.apache.hudi.cli.commands.TestArchivedCommitsCommand.testShowArchivedCommits  Time elapsed: 2.116 s  <<< ERROR!
java.net.BindException: Problem binding to [localhost:42364] java.net.BindException: Address already in use; For more details see:  http://wiki.apache.org/hadoop/BindException
	at org.apache.hudi.cli.commands.TestArchivedCommitsCommand.init(TestArchivedCommitsCommand.java:58)
Caused by: java.net.BindException: Address already in use
	at org.apache.hudi.cli.commands.TestArchivedCommitsCommand.init(TestArchivedCommitsCommand.java:58)

Don't see anything related to the changes introduced by this PR. Can we retry the failed travis job?

cc @hddong in case this is a sign of some hardcode ports etc.

cc @hddong in case this is a sign of some hardcode ports etc.

I did look into the cli test code, and not found the hardcode ports and restarted the Travis three times, it is occured occasionally

@vinothchandar  as @leesf said, it's occured occasionally .  I think It maybe due to miniDFS, it has service which use random port (like NameNode RPC) when starting.

I am OK with the changes, as discussed above, any other concern here? cc @vinothchandar


[BUG] Problem with using Hub on Windows

🐛🐛 Bug Report
⚗ Current Behavior
Issue is pertaining to a windows env. The package 'pwd' is unavailable for windows and instead a package called 'winpwd' is used. This causes the basic example of the READ.ME file to fail. It fails while instantiating the Dataset object.
⚙ Environment

OS:  Windows 10 Version 2004 Build 19041.572

Hi, I would like to work on this

Hi, I would like to work on this

FYI this is an issue with anaconda's posixpath.py.

FYI this is an issue with anaconda's posixpath.py.

Hey @madhucharan checking in on this branch, let me or Abhinav know if you hit a roadblock!

Hey @madhucharan checking in on this branch, let me or Abhinav know if you hit a roadblock!

Do you think it'd be better if everything's switched to os.path instead of posixpath?

Do you think it'd be better if everything's switched to os.path instead of posixpath?

@FayazRahman Could you expand on that suggestion? Anaconda (not Hub) calls posixpath.py, which in turn tries to import pwd on a windows machine, thus throwing an importerror.

@FayazRahman Could you expand on that suggestion? Anaconda (not Hub) calls posixpath.py, which in turn tries to import pwd on a windows machine, thus throwing an importerror.

Hub imports and uses posixpath for handling paths and the error was thrown when it tried to do posixpath.expanduser("~/.activeloop/store") because expanduser calls import pwd. Anaconda wasn't involved in my case and it was just a result of using posixpath which is meant for linux. I think using os.path would avoid future conflicts although no errors are thrown as of now.

Hub imports and uses posixpath for handling paths and the error was thrown when it tried to do posixpath.expanduser("~/.activeloop/store") because expanduser calls import pwd. Anaconda wasn't involved in my case and it was just a result of using posixpath which is meant for linux. I think using os.path would avoid future conflicts although no errors are thrown as of now.

Resolved by #472

Resolved by #472


fix(dropdown,rating,typeahead): remove global in order to be usable …

…with systemjs and aot

and here https://github.com/valor-software/ng2-bootstrap/pull/1734/files#diff-a4f2d9fe15a93898cef7b2fcb7adc4dbL45
try to replace with Event

why?

Codecov Report

Merging #1734 into development will decrease coverage by 0.03%.
The diff coverage is n/a.

@@               Coverage Diff               @@
##           development    #1734      +/-   ##
===============================================
- Coverage        85.44%   85.41%   -0.04%
===============================================
  Files               85       85
  Lines             2295     2290       -5
  Branches           300      300
===============================================
- Hits              1961     1956       -5
  Misses             235      235
  Partials            99       99

Impacted Files
Coverage Δ

src/dropdown/dropdown-toggle.directive.ts
100% <ø> (ø)
:white_check_mark:

src/dropdown/dropdown.service.ts
82% <ø> (-0.7%)
:x:

src/rating/rating.component.ts
80.3% <ø> (-0.3%)
:x:

src/typeahead/typeahead.directive.ts
76.27% <ø> (-0.14%)
:x:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 27a0229...3e30eec. Read the comment docs.

If not replace with Event it will throw an error at universal

Thanks

@valorkin could you please provide a new version?


"Encountered context mismatch" trying to debug RegExp-using application

I don't have a minimised test case for this but I can reproduce it reliably:
Run through these steps to get yourself a Flutter engine repo:
https://github.com/flutter/engine/blob/master/CONTRIBUTING.md
You shouldn't need to actually build an engine but you may need to at least get the dependencies.
Fetch this branch: https://github.com/Hixie/sky_engine/tree/debugger-crash
Open a terminal in the tools/licenses directory of that repo.
run dart --checked --observe lib/main.dart ../..
It will say "Preparing data structures..." for a while, then "Collecting licenses...", then it will show a progress count. As soon as it shows a progress count (no need to rush, that count will take hours to complete), load up http://localhost:8181/ and click "debug".
The VM crashes with:
-------------------------
Encountered context mismatch
        ctx_slot: 1
        frame_ctx_level: 0
        var_ctx_level: 0

-------------------------
Current frame:
[ Frame pc(0x7fc95924eaa5) fp(0x7fc9f8f45a30) sp(0x7fc9f8f459e8)
        function = <_findLicenseBlocks@17395340_sync_body>
        url = .../engine/src/tools/licenses/lib/licenses.dart
        line = 522
        context = Context num_variables: 1 parent:{ Context num_variables: 7 }
        context level = 0 ]

-------------------------
Context contents:
        Context@0x7fc9d87f5c09 vars(1) {
          [0] = Instance of '_AllMatchesIterator'
          Context@0x7fc9d87f5b99 vars(7) {
            [0] = [first 50 chars:] // Copyright 2015 The Chromium Authors. All rights...
	    [1] = [first 50 chars:] RegExp: pattern=((?:[-;#<!/* ]*(?:REM[-;#<!/* ]*)?...
            [2] = 1
            [3] = 2
            [4] = true
            [5] = -1
            [6] = null
          }
        }
-------------------------
Debugger stack trace...

#0000 [ Frame pc(0x7fc9e3faf077) fp(0x7fc9f8f458e0) sp(0x7fc9f8f458c0)
        function = Int32List.[]
        url = dart:typed_data
        line = 1393
        context = Context num_variables: 0
        context level = 0 ]
#0001 [ Frame pc(0x7fc9e27006a5) fp(0x7fc9f8f45988) sp(0x7fc9f8f458f0)
        function = RegExp.:matcher: ((?:[-;#<!/* ]*(?:REM[-;#<!/* ]*)?[-;#<!/*]+)?)( *)The .+ is licensed pursuant to the OpenSSL open source *\n^(?:\1|\2| )* license provided (?:below|above)\.
        url = dart:core/regexp.dart
        line = -1
        context = Context num_variables: 0
        context level = 0 ]
#0002 [ Frame pc(0x7fc97f38c156) fp(0x7fc9f8f459d8) sp(0x7fc9f8f45998)
        function =<EMAIL_ADDRESS>        url = dart:core-patch/regexp_patch.dart
        line = 268
        context = Context num_variables: 0
        context level = 0 ]
...

I can provide a full crash log on request.

@crelier @a-siva

Also @turnidge FYI

Not knowing the proper git incantation to fetch your branch, I just downloaded the zip file and expanded it on top of my flutter engine.
When running the command  dart --checked --observe lib/main.dart ../..
I get this error immediately:
Load Error for "package:archive/archive.dart": Could not resolve a package location for base at file:///usr/local/google/home/regis/flutter/engine/src/tools/licenses/lib/main.dart
The dart executable comes from here:
regis@belenos:~/flutter/engine/src/tools/licenses$ which dart
/usr/local/google/home/regis/flutter/engine/src/third_party/dart-sdk/dart-sdk/bin/dart
Am I supposed to point the packages root to somewhere?

Oh, sorry, missed a step. Run pub get in tools/licenses first.

Thanks! That helped. I can see the progress count, so I need to be patient now.
I got this error while running pub get, which I am not sure I can safely ignore:
A transformer imported unknown package "args" (in "package:args/args.dart").

Might be related: https://github.com/dart-lang/sdk/issues/26948

Thanks! Indeed, it could be related.
Adding @mhausner
I am adding the full stack trace which may be helpful in debugging:

Debugger stack trace...
#0000 [ Frame pc(0x7fb15374d203) fp(0x7fb153ffd920) sp(0x7fb153ffd900)
function =<EMAIL_ADDRESS>url = dart:typed_data-patch/typed_data.dart
line = 1187
context = Context num_variables: 0
context level = 0 ]
#0001 [ Frame pc(0x7fb0bb7c2463) fp(0x7fb153ffd9d0) sp(0x7fb153ffd930)
function = RegExp.:irregexp: ^((?:[-;#<!/* ](?:REM[-;#<!/ ])?[-;#<!/]+)?)( *)Permission *(?:\n\1\2)? *is *(?:\n\1\2)? *hereby *(?:\n\1\2)? *granted, *(?:\n\1\2)? *free *(?:\n\1\2)? *of *(?:\n\1\2)? *charge, *(?:\n\1\2)? *to *(?:\n\1\2)? *any *(?:\n\1\2)? *person *(?:\n\1\2)? *obtaining *(?:\n\1\2)? *a *(?:\n\1\2)? *copy *(?:\n\1\2)? *of *(?:\n\1\2)? *this *(?:\n\1\2)? *software *(?:\n\1\2)? *and *(?:\n\1\2)? *associated *(?:\n\1\2)? *documentation *(?:\n\1\2)? *files *(?:\n\1\2)? *(the *(?:\n\1\2)? *"Software"), *(?:\n\1\2)? *to *(?:\n\1\2)? *deal *(?:\n\1\2)? *in *(?:\n\1\2)? *the *(?:\n\1\2)? *Software *(?:\n\1\2)? *without *(?:\n\1\2)? *restriction, *(?:\n\1\2)? *including *(?:\n\1\2)? *without *(?:\n\1\2)? *limitation *(?:\n\1\2)? *the *(?:\n\1\2)? *rights *(?:\n\1\2)? *to *(?:\n\1\2)? *use, *(?:\n\1\2)? *copy, *(?:\n\1\2)? *modify, *(?:\n\1\2)? *merge, *(?:\n\1\2)? *publish, *(?:\n\1\2)? *distribute, *(?:\n\1\2)? *sublicense, *(?:\n\1\2)? *and/or *(?:\n\1\2)? *sell *(?:\n\1\2)? *copies *(?:\n\1\2)? *of *(?:\n\1\2)? *the *(?:\n\1\2)? *Software, *(?:\n\1\2)? *and *(?:\n\1\2)? *to *(?:\n\1\2)? *permit *(?:\n\1\2)? *persons *(?:\n\1\2)? *to *(?:\n\1\2)? *whom *(?:\n\1\2)? *the *(?:\n\1\2)? *Software *(?:\n\1\2)? *is *(?:\n\1\2)? *furnished *(?:\n\1\2)? *to *(?:\n\1\2)? *do *(?:\n\1\2)? *so, *(?:\n\1\2)? *subject *(?:\n\1\2)? *to *(?:\n\1\2)? *the *(?:\n\1\2)? *following *(?:\n\1\2)? *conditions:(?:(?:\1\2? *)? *\n)*The *(?:\n\1\2)? *above *(?:\n\1\2)? *copyright *(?:\n\1\2)? *notice *(?:\n\1\2)? *and *(?:\n\1\2)? *this *(?:\n\1\2)? *permission *(?:\n\1\2)? *notice *(?:\n\1\2)? *shall *(?:\n\1\2)? *be *(?:\n\1\2)? *included *(?:\n\1\2)? *in *(?:\n\1\2)? *all *(?:\n\1\2)? *copies *(?:\n\1\2)? *or *(?:\n\1\2)? *substantial *(?:\n\1\2)? *portions *(?:\n\1\2)? *of *(?:\n\1\2)? *the *(?:\n\1\2)? *Software.(?:(?:\1\2? *)? *\n)*THE *(?:\n\1\2)? *SOFTWARE *(?:\n\1\2)? *IS *(?:\n\1\2)? *PROVIDED *(?:\n\1\2)? *"AS *(?:\n\1\2)? *IS", *(?:\n\1\2)? *WITHOUT *(?:\n\1\2)? *WARRANTY *(?:\n\1\2)? *OF *(?:\n\1\2)? *ANY *(?:\n\1\2)? *KIND, *(?:\n\1\2)? *EXPRESS *(?:\n\1\2)? *OR *(?:\n\1\2)? *IMPLIED, *(?:\n\1\2)? *INCLUDING *(?:\n\1\2)? *BUT *(?:\n\1\2)? *NOT *(?:\n\1\2)? *LIMITED *(?:\n\1\2)? *TO *(?:\n\1\2)? *THE *(?:\n\1\2)? *WARRANTIES *(?:\n\1\2)? *OF *(?:\n\1\2)? *MERCHANTABILITY, *(?:\n\1\2)? *FITNESS *(?:\n\1\2)? *FOR *(?:\n\1\2)? *A *(?:\n\1\2)? *PARTICULAR *(?:\n\1\2)? *PURPOSE *(?:\n\1\2)? *AND *(?:\n\1\2)? *NONINFRINGEMENT. *(?:\n\1\2)? *IN *(?:\n\1\2)? *NO *(?:\n\1\2)? *EVENT *(?:\n\1\2)? *SHALL *(?:\n\1\2)? *THE *(?:\n\1\2)? *AUTHORS *(?:\n\1\2)? *BE *(?:\n\1\2)? *LIABLE *(?:\n\1\2)? *FOR *(?:\n\1\2)? *ANY *(?:\n\1\2)? *CLAIM, *(?:\n\1\2)? *DAMAGES *(?:\n\1\2)? *OR *(?:\n\1\2)? *OTHER *(?:\n\1\2)? *LIABILITY, *(?:\n\1\2)? *WHETHER *(?:\n\1\2)? *IN *(?:\n\1\2)? *AN *(?:\n\1\2)? *ACTION *(?:\n\1\2)? *OF *(?:\n\1\2)? *CONTRACT, *(?:\n\1\2)? *TORT *(?:\n\1\2)? *OR *(?:\n\1\2)? *OTHERWISE, *(?:\n\1\2)? *ARISING *(?:\n\1\2)? *FROM, *(?:\n\1\2)? *OUT *(?:\n\1\2)? *OF *(?:\n\1\2)? *OR *(?:\n\1\2)? *IN *(?:\n\1\2)? *CONNECTION *(?:\n\1\2)? *WITH *(?:\n\1\2)? *THE *(?:\n\1\2)? *SOFTWARE *(?:\n\1\2)? *OR *(?:\n\1\2)? *THE *(?:\n\1\2)? *USE *(?:\n\1\2)? *OR *(?:\n\1\2)? *OTHER *(?:\n\1\2)? *DEALINGS *(?:\n\1\2)? *IN *(?:\n\1\2)? *THE *(?:\n\1\2)? *SOFTWARE.
url = dart:core/regexp.dart
line = -1
context = Context num_variables: 0
context level = 0 ]
#0002 [ Frame pc(0x7fb153873f89) fp(0x7fb153ffda20) sp(0x7fb153ffd9e0)
function =<EMAIL_ADDRESS>url = dart:core-patch/regexp_patch.dart
line = 268
context = Context num_variables: 0
context level = 0 ]
#0003 [ Frame pc(0x7fb0c20f59a6) fp(0x7fb153ffda80) sp(0x7fb153ffda30)
function = <_findLicenseBlocks@35923893_sync_body>
url = file:///usr/local/google/home/regis/flutter/engine/src/tools/licenses/lib/licenses.dart
line = 522
context = Context num_variables: 1 parent:{ Context num_variables: 7 }
context level = 0 ]
#0004 [ Frame pc(0x7fb153e7a9f8) fp(0x7fb153ffdad8) sp(0x7fb153ffda90)
function =<EMAIL_ADDRESS>url = dart:core-patch/core_patch.dart
line = 61
context = Context num_variables: 0
context level = 0 ]
#0005 [ Frame pc(0x7fb0c1be8a6f) fp(0x7fb153ffdb10) sp(0x7fb153ffdae8)
function = <_tryInline@35923893_sync_body>
url = file:///usr/local/google/home/regis/flutter/engine/src/tools/licenses/lib/licenses.dart
line = 704
context = Context num_variables: 1 parent:{ Context num_variables: 5 }
context level = 0 ]
#0006 [ Frame pc(0x7fb153e7a9f8) fp(0x7fb153ffdb68) sp(0x7fb153ffdb20)
function =<EMAIL_ADDRESS>url = dart:core-patch/core_patch.dart
line = 61
context = Context num_variables: 0
context level = 0 ]
#0007 [ Frame pc(0x7fb13261a27a) fp(0x7fb153ffdbd0) sp(0x7fb153ffdb78)
function = ExpandIterator.moveNext
url = dart:_internal/iterable.dart
line = 472
context = Context num_variables: 0
context level = 0 ]
#0008 [ Frame pc(0x7fb1326192ad) fp(0x7fb153ffdc48) sp(0x7fb153ffdbe0)
function =<EMAIL_ADDRESS>url = dart:core-patch/growable_array.dart
line = 182
context = Context num_variables: 0
context level = 0 ]
#0009 [ Frame pc(0x7fb13261d40a) fp(0x7fb153ffdcb8) sp(0x7fb153ffdc58)
function = determineLicensesFor
url = file:///usr/local/google/home/regis/flutter/engine/src/tools/licenses/lib/licenses.dart
line = 730
context = Context num_variables: 4
context level = 1 ]
#0010 [ Frame pc(0x7fb13262835f) fp(0x7fb153ffdd28) sp(0x7fb153ffdcc8)
function = RepositorySourceFile.get:licenses
url = file:///usr/local/google/home/regis/flutter/engine/src/tools/licenses/lib/main.dart
line = 82
context = Context num_variables: 1
context level = 1 ]
#0011 [ Frame pc(0x7fb132631a7b) fp(0x7fb153ffdda0) sp(0x7fb153ffdd38)
function = RepositoryDirectory.getLicenses
url = file:///usr/local/google/home/regis/flutter/engine/src/tools/licenses/lib/main.dart
line = 938
context = Context num_variables: 0
context level = 0 ]
#0012 [ Frame pc(0x7fb1326315b0) fp(0x7fb153ffde28) sp(0x7fb153ffddb0)
function = RepositoryDirectory.getLicenses
url = file:///usr/local/google/home/regis/flutter/engine/src/tools/licenses/lib/main.dart
line = 934
context = Context num_variables: 0
context level = 0 ]
#0013 [ Frame pc(0x7fb1326315b0) fp(0x7fb153ffdeb0) sp(0x7fb153ffde38)
function = RepositoryDirectory.getLicenses
url = file:///usr/local/google/home/regis/flutter/engine/src/tools/licenses/lib/main.dart
line = 934
context = Context num_variables: 0
context level = 0 ]
#0014 [ Frame pc(0x7fb1326315b0) fp(0x7fb153ffdf38) sp(0x7fb153ffdec0)
function = RepositoryDirectory.getLicenses
url = file:///usr/local/google/home/regis/flutter/engine/src/tools/licenses/lib/main.dart
line = 934
context = Context num_variables: 0
context level = 0 ]
#0015 [ Frame pc(0x7fb153e4eea6) fp(0x7fb153ffdfa8) sp(0x7fb153ffdf48)
function = main
url = file:///usr/local/google/home/regis/flutter/engine/src/tools/licenses/lib/main.dart
line = 1477
context = Context num_variables: 3 parent:{ Context num_variables: 0 }
context level = 1 ]
#0016 [ Frame pc(0x7fb153e4e3c4) fp(0x7fb153ffe068) sp(0x7fb153ffe038)
function =
url = dart:isolate-patch/isolate_patch.dart
line = 259
context = Context num_variables: 5
context level = 0 ]
#0017 [ Frame pc(0x7fb15396de47) fp(0x7fb153ffe0f0) sp(0x7fb153ffe0c0)
function = _RawReceivePortImpl@709387912._handleMessage@709387912
url = dart:isolate-patch/isolate_patch.dart
line = 148
context = Context num_variables: 0
context level = 0 ]

@Hixie Can you confirm that you also saw an issue where regular expressions stopped making progress? We might need to block Dart 1.18 release on this issue.

I thought I did, but further study suggested I just had pathological expressions and they were just taking forever to complete. (That's why I was trying to connect the debugger, to find out what expression was the problem.)

Unfortunately, the fix for #26948 does not fix this issue.
In debug mode, the debugger crashes with an assert fault before reporting the context mismatch:
runtime/vm/object.cc: 237: error: expected: strlen(cname) == static_cast<size_t>(name.Length())
It looks like the debugger is having trouble scrubbing a name containing a regexp. At least, the length is not consistent (e.g. 6478 vs 6482). The same inconsistency exists in release mode, before the context mismatch is reported. But I verified that the mismatch is not a consequence of scrubbing the name.
Let me dig deeper...


Add migration popup for v1 -> v3 projectIds on relaunch

What does this PR do and why?

Screenshots or screen recordings
If applicable, provide screenshots or screen recordings to demonstrate the changes.
Acceptance checklist

[ ] I have evaluated the Approval Guidelines for this PR.
[ ] I have tested this PR in all supported browsers.
[ ] I have tested this PR in dark mode and light mode (if applicable).

Current dependencies on/for this PR:

main

PR #2623

PR #2671

PR #2672   👈

This comment was auto-generated by Graphite.

Graphite Merge Job
Current status: ⏳ Queued to merge
This pull request is currently queued to be merged as part of a stack.
This comment was auto-generated by Graphite.

Job Reference: bNgIkwBqux2UJLmdY4Uy

Graphite Merge Job
Current status: 🔄 Waiting for checks to pass
This pull request is currently waiting for CI and mergeability checks to pass before being merged.
This comment was auto-generated by Graphite.

Job Reference: nnbd1Zy8vknQYYi7LCtZ


SSO cache file hash incorrect

Describe the bug
When boto tries to load sso credentials from the JSON cache file, it generates a hash that doesn't match the filenames. It is looking for the file 59146b57f9acd0df73c13bbc6a53478be4ca6ca0.json but the login command (aws sso login) generates different filenames.
Expected Behavior
Boto3 should be able to load the cached SSO credentials correctly.
Current Behavior
Running any boto example returns the following exception:
Traceback (most recent call last):
  File "/Users/username/Documents/code/pyscript/venv/lib/python3.10/site-packages/botocore/credentials.py", line 322, in __getitem__
    with open(actual_key) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/Users/username/.aws/sso/cache/59146b57f9acd0df73c13bbc6a53478be4ca6ca0.json'

...

botocore.exceptions.SSOTokenLoadError: Error loading SSO Token: The SSO access token has either expired or is otherwise invalid.

Reproduction Steps
Login via SSO: aws sso login
Ensure that the cli works: aws s3 ls
Run a python example (hello amazon S3): https://docs.aws.amazon.com/code-library/latest/ug/python_3_s3_code_examples.html
Possible Solution
Fix the hash algorithm in credentials.py [SSOCredentialFetcher._create_cache_key(self)] to match however the AWS cli creates the SSO cache filenames.
Additional Information/Context
No response
SDK version used
boto3: 1.24.59, botocore: 1.27.59
Environment details (OS name and version, etc.)
osx ventura, Version 13.0.1 (22A400)
python 3.10.4

Hi @bisrael8191 thanks for reaching out. Here is the boto3 documentation for using SSO (now IAM Identity Center): https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html#aws-iam-identity-center.
Also referenced in that documentation is the AWS CLI documentation for configuring SSO: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sso.html.
I think the first thing to rule out is that you have correctly configured SSO. In order to investigate further could you please share your debug logs (with any sensitive info redacted) by adding boto3.set_stream_logger('') to your script? I found another issue that might be related to this one, with more context on the caching behavior provided here.


Don't call t.Fatal from a goroutine 💀 & pin prow-tests 📌

Changes
First issue
It turns out that calling t.Fatal (and other functions that end tests,
see https://tip.golang.org/pkg/testing/#T) is only safe from the
goroutine running the Test function, so when this test started to fail
(root cause still TBD) (e.g. in
https://github.com/tektoncd/pipeline/pull/654) we started to see race
condition failures, kinda masking whatever was actually going wrong.
I think I caught all the instances of this but not 100% sure - note that
using it in sub tests (https://blog.golang.org/subtests) is a-ok :D
Second issue
We were using stable and stable was suddenly updated to install ko
from https://github.com/google/ko but unfortunately that version of ko
didn't include google/go-containerregistry#380
which was the critical change from building binaries at /ko-app to
building them at /ko-app/my-pkg <- which we built some of our
functionality around.
Now we'll try to pin the image so we aren't caught by surprise by
changes again and can decide for ourselves when we want to consume them.
(Hopefully old images don't get deleted too often.. 🙏 Or at least not
before #267 😛)
I'm gonna manually apply this right now 🤞🤞🤞
Submitter Checklist
These are the criteria that every PR should meet, please check them off as you
review them:

[n/a] Includes tests (if functionality changed/added)
[n/a] Includes docs (if user facing)
[x] Commit messages follow commit message best practices

See the contribution guide for more details.
Release Notes
n/a

/test pull-tekton-pipeline-integration-tests

Second issue
We were using stable and stable was suddenly updated to install ko
from https://github.com/google/ko but unfortunately that version of ko
didn't include google/go-containerregistry#380
which was the critical change from building binaries at /ko-app to
building them at /ko-app/my-pkg <- which we built some of our
functionality around.

@ImJasonH is this right ? we want google/go-containerregistry#380 in google/ko :angel:  :sweat_smile:

Now we'll try to pin the image so we aren't caught by surprise by
changes again and can decide for ourselves when we want to consume them.
(Hopefully old images don't get deleted too often.. pray Or at least not
before #267 stuck_out_tongue)

:crossed_fingers:


⚠ Home NAS has degraded performance

In d689d93, Home NAS ($MY_NAS_URL) experienced degraded performance:

HTTP code: 200
Response time: 7314 ms

Resolved: Home NAS performance has improved in 24241eb.


normaliz: init at 3.10.1

Description of changes
Normaliz is an open source tool for computations in affine monoids, vector configurations, lattice polytopes, and rational cones.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Hello @yannickulrich, welcome to nixpkgs!
I've had a look at upstream, and this package currently seem to be missing a lot of optional dependencies, which seems to reduce the amount of features in the program by quite a bit? I don't know this program, so do you know to what degree these features should be considered optional, or whether this is core functionality? (most of these aren't trivial to add, because they aren't packaged yet)

thanks for you review, @h7x4 & @AndersonTorres.
I've mostly used the features of normaliz that were available. Adding nauty and flit is trivial as both are already packaged. Looking at the code, this would add features. However, for arb and antic (both packaged) to be useful, we'd also need need e-antic which isn't yet packaged. I'm not sure what the correct procedures are but I'd suggest adding nautry & flint now and the others later.

Never mind, forgot to test properly. Will update again in a sec

Sorry for missing this. It looks good!
@ofborg eval


Node stops producing blocks after transaction from new accounts

This is related to https://github.com/0xPolygonMiden/miden-node/issues/176.
After #200 was merged, submitting proofs for new accounts works fine on the client, which means the RPC does not return an error:
Running `target/release/miden-client tx new consume-notes 0x444bca3cbb5d302d 0xbd8f70675194683eec7b8f8d5b29c7306970dcaaca44726dac9c07624fafa2ae`
2024-02-05T18:03:34.138529Z  INFO miden_client::cli::transactions: Executed transaction, proving and then submitting...
2024-02-05T18:03:34.190356Z  INFO Proving program: miden_prover: Generated execution trace of 70 columns and 131072 steps (27% padded) in 51 ms
2024-02-05T18:03:35.077097Z  INFO miden_client::client::transactions: Proved transaction, submitting to the node...
2024-02-05T18:03:35.115166Z  INFO miden_client::store::transactions: Transaction id 0x134e33c69d0db99f744922a98b4a54edb85fc58862243f0c91afddafc0bcba85
2024-02-05T18:03:35.115174Z  INFO miden_client::store::transactions: Transaction account id: 0x444bca3cbb5d302d

However, after the proof is actually submitted, the node is then locked in a state where it cannot generate blocks, and fails periodically with the following error:
2024-02-05T18:05:09.573085Z ERROR block_producer:try_build_block:build_block: miden-block-producer: block-producer/src/block_builder/mod.rs:73: error: transaction batches and store contain different hashes for some accounts. Offending accounts: [AccountId(4921249380022693933)]

...

2024-02-05T18:10:19.571170Z ERROR block_producer:try_build_block:build_block: miden-block-producer: block-producer/src/block_builder/mod.rs:73: error: transaction batches and store contain different hashes for some accounts. Offending accounts: [AccountId(4921249380022693933)]

This does not happen when executing transactions against accounts that have already been added to the accounts tree (like the ones generated on genesis).
Seems like there might be a check on the block producer that does not take the newer changes from #200 into account.

If you look further into the error that you copied you will see that the problem here is when we are building the block_witness we are making a certain number of checks and one of them is not accounting for new accounts. It is comparing the received account hash and the one in store and throwing an error if they are different. Obviously for a new account the store one would be Word 0 and the received one would be Word X, hence erroring out.

This is happening somewhere here, right? If so, the fix should be similar to what we did in #200.

After fixing this issue I found another one that I am investigating at the moment where block proving doesn't work anymore because of the fixes to accept new accounts.

What kind of an error are you getting?

I have also found multiple situations where the node arrives to a deadlock. Will need to investigate further in the future because there are definitely hidden bugs.

I think the main issue right now is that the node does not recover from "bad transactions". Basically, if some transaction causes issues with block building (for w/e reason), the node has no mechanism for discarding it. That is, it will just keep trying to add it to the block. The assumption here is that if a transaction made it into the transaction queue, it should not cause issues downstream - but in this case because of a separate bug, this is not the case. Let's create a separate issue to figure out how we want to deal with such situations.

but in this case because of a separate bug, this is not the case. Let's create a separate issue to figure out how we want to deal with such situations.

Could you be clearer I have not fully understood your point please.

One of the problems we have is that if there is a transaction in the transaction queue which causes block production to fail, the node will deadlock (it will try to create a block with the same transaction over and over again). In this case, we know how to fix the problem (i.e., the transaction in question should actually not fail), but it would be good to have a more general solution for such situations (i.e., to avoid deadlocks even if there are bugs).

What kind of an error are you getting?

�[2m2024-02-05T12:46:00.031022Z�[0m �[31mERROR�[0m �[1mblock_producer�[0m:�[1mtry_build_block�[0m:�[1mbuild_block�[0m: �[2mmiden-block-producer�[0m�[2m:�[0m �[2mblock-producer/src/block_builder/mod.rs�[0m�[2m:�[0m�[2m73�[0m�[2m:�[0m �[1merror�[0m: failed to compute new block: program execution failed
After submitting a transaction using new accounts no further blocks can be proven.

This is likely due to some assumption within the MASM file which we use to compute the new block header being no longer valid.

I believe after testing that this issue has been resolved. I have not seen it arise again.
These PR should have fixed it:

https://github.com/0xPolygonMiden/miden-node/pull/200
https://github.com/0xPolygonMiden/miden-node/pull/230
https://github.com/0xPolygonMiden/miden-node/pull/232

Opened this following issue for improved transaction handling and queue management as discussed with @bobbinth and @hackaugusto on the call Monday: https://github.com/0xPolygonMiden/miden-node/issues/235


Make okhttp and apollo api deps in http-cache

These are both part of the cache's public API, and gives warnings in the IDE when they're hidden from the classpath.

You'll need to apply the java-library plugin for api to be available.

To fix the build or you think that’s an undue requirement?

I guess it's a reasonable addition, not sure why it's not there already actually.


No TLSHANDSHAKETIMESHIFT for intermediate Ca Certificates

Hy everyone,
I'm having a problem with expired certificates, I'm running a cluster with several channels and I updated the certificates on time for most of them, but I forgot about a channel that was created and used only during the first initialization of the cluster.
So today in my test environment the orderers got restarted and now they don't start, my production environment has the same problem but as they haven't been restarted they are running as if nothing has changed, for now.
So, I did as it is said, in this issue and changed these environment variables:

ORDERER_GENERAL_AUTHENTICATION_NOEXPIRATIONCHECKS: "true"
ORDERER_GENERAL_CLUSTER_TLSHANDSHAKETIMESHIFT: 96h
ORDERER_GENERAL_TLS_TLSHANDSHAKETIMESHIFT: 96h
ORDERER_OPERATIONS_TLS_TLSHANDSHAKETIMESHIFT: 96h
ORDERER_ADMIN_TLS_TLSHANDSHAKETIMESHIFT: 96h

The only handshake time shift still missing is the one from kafta that I don't use, I'm using Raft, but still, I'm getting this error trying to run the orderers:
PANI 00d Error creating ledger resources: error creating channelconfig bundle: initializing channelconfig failed: could not create channel Orderer sub-group config: setting up the MSP manager failed: CA Certificate is not valid, (SN:<PHONE_NUMBER>81241201287378615618194486727802882): could not obtain certification chain: the supplied identity is not valid: x509: certificate has expired or is not yet valid: current time 2022-02-17T14:28:20Z is after 2022-02-16T16:41:01Z
I published the issue in the message channel and with the help of @yacovm found that there's no TLS Handshake time shift for the CA certificates.
Here's the stack trace:
Thu, Feb 17 2022 9:28:20 am	2022-02-17 14:28:20.681 UTC [localconfig] completeInitialization -> WARN 001 General.GenesisFile should be replaced by General.BootstrapFile
Thu, Feb 17 2022 9:28:20 am	2022-02-17 14:28:20.681 UTC [localconfig] completeInitialization -> INFO 002 Kafka.Version unset, setting to <IP_ADDRESS>
Thu, Feb 17 2022 9:28:20 am	2022-02-17 14:28:20.681 UTC [orderer.common.server] prettyPrintStruct -> INFO 003 Orderer config values:
Thu, Feb 17 2022 9:28:20 am	General.ListenAddress = "<IP_ADDRESS>"
Thu, Feb 17 2022 9:28:20 am	General.ListenPort = 7050
Thu, Feb 17 2022 9:28:20 am	General.TLS.Enabled = true
Thu, Feb 17 2022 9:28:20 am	General.TLS.PrivateKey = "/var/hyperledger/orderer/tls/server.key"
Thu, Feb 17 2022 9:28:20 am	General.TLS.Certificate = "/var/hyperledger/orderer/tls/server.crt"
Thu, Feb 17 2022 9:28:20 am	General.TLS.RootCAs = [/var/hyperledger/orderer/tls/chain.crt]
Thu, Feb 17 2022 9:28:20 am	General.TLS.ClientAuthRequired = true
Thu, Feb 17 2022 9:28:20 am	General.TLS.ClientRootCAs = [/var/hyperledger/orderer/tls/chain.crt]
Thu, Feb 17 2022 9:28:20 am	General.TLS.TLSHandshakeTimeShift = 96h0m0s
Thu, Feb 17 2022 9:28:20 am	General.Cluster.ListenAddress = ""
Thu, Feb 17 2022 9:28:20 am	General.Cluster.ListenPort = 0
Thu, Feb 17 2022 9:28:20 am	General.Cluster.ServerCertificate = ""
Thu, Feb 17 2022 9:28:20 am	General.Cluster.ServerPrivateKey = ""
Thu, Feb 17 2022 9:28:20 am	General.Cluster.ClientCertificate = "/var/hyperledger/orderer/tls/server.crt"
Thu, Feb 17 2022 9:28:20 am	General.Cluster.ClientPrivateKey = "/var/hyperledger/orderer/tls/server.key"
Thu, Feb 17 2022 9:28:20 am	General.Cluster.RootCAs = [/var/hyperledger/orderer/tls/chain.crt]
Thu, Feb 17 2022 9:28:20 am	General.Cluster.DialTimeout = 5s
Thu, Feb 17 2022 9:28:20 am	General.Cluster.RPCTimeout = 7s
Thu, Feb 17 2022 9:28:20 am	General.Cluster.ReplicationBufferSize = 20971520
Thu, Feb 17 2022 9:28:20 am	General.Cluster.ReplicationPullTimeout = 5s
Thu, Feb 17 2022 9:28:20 am	General.Cluster.ReplicationRetryTimeout = 5s
Thu, Feb 17 2022 9:28:20 am	General.Cluster.ReplicationBackgroundRefreshInterval = 5m0s
Thu, Feb 17 2022 9:28:20 am	General.Cluster.ReplicationMaxRetries = 12
Thu, Feb 17 2022 9:28:20 am	General.Cluster.SendBufferSize = 10
Thu, Feb 17 2022 9:28:20 am	General.Cluster.CertExpirationWarningThreshold = 168h0m0s
Thu, Feb 17 2022 9:28:20 am	General.Cluster.TLSHandshakeTimeShift = 96h0m0s
Thu, Feb 17 2022 9:28:20 am	General.Keepalive.ServerMinInterval = 1m0s
Thu, Feb 17 2022 9:28:20 am	General.Keepalive.ServerInterval = 2h0m0s
Thu, Feb 17 2022 9:28:20 am	General.Keepalive.ServerTimeout = 20s
Thu, Feb 17 2022 9:28:20 am	General.ConnectionTimeout = 0s
Thu, Feb 17 2022 9:28:20 am	General.GenesisMethod = "file"
Thu, Feb 17 2022 9:28:20 am	General.GenesisFile = "/var/hyperledger/orderer/genesis.block"
Thu, Feb 17 2022 9:28:20 am	General.BootstrapMethod = "file"
Thu, Feb 17 2022 9:28:20 am	General.BootstrapFile = "/var/hyperledger/orderer/genesis.block"
Thu, Feb 17 2022 9:28:20 am	General.Profile.Enabled = false
Thu, Feb 17 2022 9:28:20 am	General.Profile.Address = "<IP_ADDRESS>:6060"
Thu, Feb 17 2022 9:28:20 am	General.LocalMSPDir = "/var/hyperledger/orderer/msp"
Thu, Feb 17 2022 9:28:20 am	General.LocalMSPID = "company"
Thu, Feb 17 2022 9:28:20 am	General.BCCSP.Default = "SW"
Thu, Feb 17 2022 9:28:20 am	General.BCCSP.SW.Security = 256
Thu, Feb 17 2022 9:28:20 am	General.BCCSP.SW.Hash = "SHA2"
Thu, Feb 17 2022 9:28:20 am	General.BCCSP.SW.FileKeystore.KeyStorePath = ""
Thu, Feb 17 2022 9:28:20 am	General.Authentication.TimeWindow = 15m0s
Thu, Feb 17 2022 9:28:20 am	General.Authentication.NoExpirationChecks = true
Thu, Feb 17 2022 9:28:20 am	General.MaxRecvMsgSize = 104857600
Thu, Feb 17 2022 9:28:20 am	General.MaxSendMsgSize = 104857600
Thu, Feb 17 2022 9:28:20 am	FileLedger.Location = "/var/hyperledger/production/orderer"
Thu, Feb 17 2022 9:28:20 am	FileLedger.Prefix = ""
Thu, Feb 17 2022 9:28:20 am	Kafka.Retry.ShortInterval = 5s
Thu, Feb 17 2022 9:28:20 am	Kafka.Retry.ShortTotal = 10m0s
Thu, Feb 17 2022 9:28:20 am	Kafka.Retry.LongInterval = 5m0s
Thu, Feb 17 2022 9:28:20 am	Kafka.Retry.LongTotal = 12h0m0s
Thu, Feb 17 2022 9:28:20 am	Kafka.Retry.NetworkTimeouts.DialTimeout = 10s
Thu, Feb 17 2022 9:28:20 am	Kafka.Retry.NetworkTimeouts.ReadTimeout = 10s
Thu, Feb 17 2022 9:28:20 am	Kafka.Retry.NetworkTimeouts.WriteTimeout = 10s
Thu, Feb 17 2022 9:28:20 am	Kafka.Retry.Metadata.RetryMax = 3
Thu, Feb 17 2022 9:28:20 am	Kafka.Retry.Metadata.RetryBackoff = 250ms
Thu, Feb 17 2022 9:28:20 am	Kafka.Retry.Producer.RetryMax = 3
Thu, Feb 17 2022 9:28:20 am	Kafka.Retry.Producer.RetryBackoff = 100ms
Thu, Feb 17 2022 9:28:20 am	Kafka.Retry.Consumer.RetryBackoff = 2s
Thu, Feb 17 2022 9:28:20 am	Kafka.Verbose = false
Thu, Feb 17 2022 9:28:20 am	Kafka.Version = <IP_ADDRESS>
Thu, Feb 17 2022 9:28:20 am	Kafka.TLS.Enabled = false
Thu, Feb 17 2022 9:28:20 am	Kafka.TLS.PrivateKey = ""
Thu, Feb 17 2022 9:28:20 am	Kafka.TLS.Certificate = ""
Thu, Feb 17 2022 9:28:20 am	Kafka.TLS.RootCAs = []
Thu, Feb 17 2022 9:28:20 am	Kafka.TLS.ClientAuthRequired = false
Thu, Feb 17 2022 9:28:20 am	Kafka.TLS.ClientRootCAs = []
Thu, Feb 17 2022 9:28:20 am	Kafka.TLS.TLSHandshakeTimeShift = 0s
Thu, Feb 17 2022 9:28:20 am	Kafka.SASLPlain.Enabled = false
Thu, Feb 17 2022 9:28:20 am	Kafka.SASLPlain.User = ""
Thu, Feb 17 2022 9:28:20 am	Kafka.SASLPlain.Password = ""
Thu, Feb 17 2022 9:28:20 am	Kafka.Topic.ReplicationFactor = 3
Thu, Feb 17 2022 9:28:20 am	Debug.BroadcastTraceDir = ""
Thu, Feb 17 2022 9:28:20 am	Debug.DeliverTraceDir = ""
Thu, Feb 17 2022 9:28:20 am	Consensus = map[SnapDir:/var/hyperledger/production/orderer/etcdraft/snapshot WALDir:/var/hyperledger/production/orderer/etcdraft/wal]
Thu, Feb 17 2022 9:28:20 am	Operations.ListenAddress = ":8443"
Thu, Feb 17 2022 9:28:20 am	Operations.TLS.Enabled = false
Thu, Feb 17 2022 9:28:20 am	Operations.TLS.PrivateKey = ""
Thu, Feb 17 2022 9:28:20 am	Operations.TLS.Certificate = ""
Thu, Feb 17 2022 9:28:20 am	Operations.TLS.RootCAs = []
Thu, Feb 17 2022 9:28:20 am	Operations.TLS.ClientAuthRequired = false
Thu, Feb 17 2022 9:28:20 am	Operations.TLS.ClientRootCAs = []
Thu, Feb 17 2022 9:28:20 am	Operations.TLS.TLSHandshakeTimeShift = 96h0m0s
Thu, Feb 17 2022 9:28:20 am	Metrics.Provider = "prometheus"
Thu, Feb 17 2022 9:28:20 am	Metrics.Statsd.Network = "udp"
Thu, Feb 17 2022 9:28:20 am	Metrics.Statsd.Address = "<IP_ADDRESS>:8125"
Thu, Feb 17 2022 9:28:20 am	Metrics.Statsd.WriteInterval = 30s
Thu, Feb 17 2022 9:28:20 am	Metrics.Statsd.Prefix = ""
Thu, Feb 17 2022 9:28:20 am	ChannelParticipation.Enabled = false
Thu, Feb 17 2022 9:28:20 am	ChannelParticipation.MaxRequestBodySize = 1048576
Thu, Feb 17 2022 9:28:20 am	Admin.ListenAddress = "<IP_ADDRESS>:9443"
Thu, Feb 17 2022 9:28:20 am	Admin.TLS.Enabled = false
Thu, Feb 17 2022 9:28:20 am	Admin.TLS.PrivateKey = ""
Thu, Feb 17 2022 9:28:20 am	Admin.TLS.Certificate = ""
Thu, Feb 17 2022 9:28:20 am	Admin.TLS.RootCAs = []
Thu, Feb 17 2022 9:28:20 am	Admin.TLS.ClientAuthRequired = true
Thu, Feb 17 2022 9:28:20 am	Admin.TLS.ClientRootCAs = []
Thu, Feb 17 2022 9:28:20 am	Admin.TLS.TLSHandshakeTimeShift = 96h0m0s
Thu, Feb 17 2022 9:28:20 am	2022-02-17 14:28:20.698 UTC [orderer.common.server] initializeServerConfig -> INFO 004 Starting orderer with mutual TLS enabled
Thu, Feb 17 2022 9:28:20 am	2022-02-17 14:28:20.730 UTC [orderer.common.server] Main -> INFO 005 Not bootstrapping the system channel because of existing channels
Thu, Feb 17 2022 9:28:20 am	2022-02-17 14:28:20.800 UTC [orderer.common.server] Main -> INFO 006 Starting without a system channel
Thu, Feb 17 2022 9:28:20 am	2022-02-17 14:28:20.800 UTC [orderer.common.server] Main -> INFO 007 Setting up cluster
Thu, Feb 17 2022 9:28:20 am	2022-02-17 14:28:20.800 UTC [orderer.common.server] reuseListener -> INFO 008 Cluster listener is not configured, defaulting to use the general listener on port 7050
Thu, Feb 17 2022 9:28:20 am	2022-02-17 14:28:20.800 UTC [orderer.common.server] reuseListener -> INFO 009 Cluster listener is not configured, defaulting to use the general listener on port 7050
Thu, Feb 17 2022 9:28:20 am	2022-02-17 14:28:20.800 UTC [certmonitor] trackCertExpiration -> INFO 00a The enrollment certificate will expire on 2023-01-25 23:25:48 +0000 UTC
Thu, Feb 17 2022 9:28:20 am	2022-02-17 14:28:20.800 UTC [certmonitor] trackCertExpiration -> INFO 00b The server TLS certificate will expire on 2023-01-25 23:25:16 +0000 UTC
Thu, Feb 17 2022 9:28:20 am	2022-02-17 14:28:20.801 UTC [certmonitor] trackCertExpiration -> INFO 00c The client TLS certificate will expire on 2023-01-25 23:25:16 +0000 UTC
Thu, Feb 17 2022 9:28:20 am	2022-02-17 14:28:20.825 UTC [orderer.commmon.multichannel] initSystemChannel -> PANI 00d Error creating ledger resources: error creating channelconfig bundle: initializing channelconfig failed: could not create channel Orderer sub-group config: setting up the MSP manager failed: CA Certificate is not valid, (SN:<PHONE_NUMBER>81241201287378615618194486727802882): could not obtain certification chain: the supplied identity is not valid: x509: certificate has expired or is not yet valid: current time 2022-02-17T14:28:20Z is after 2022-02-16T16:41:01Z
Thu, Feb 17 2022 9:28:20 am	panic: Error creating ledger resources: error creating channelconfig bundle: initializing channelconfig failed: could not create channel Orderer sub-group config: setting up the MSP manager failed: CA Certificate is not valid, (SN:<PHONE_NUMBER>81241201287378615618194486727802882): could not obtain certification chain: the supplied identity is not valid: x509: certificate has expired or is not yet valid: current time 2022-02-17T14:28:20Z is after 2022-02-16T16:41:01Z
Thu, Feb 17 2022 9:28:20 am
Thu, Feb 17 2022 9:28:20 am	goroutine 1 [running]:
Thu, Feb 17 2022 9:28:20 am	go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc0002620b0, 0x0, 0x0, 0x0)
Thu, Feb 17 2022 9:28:20 am	/go/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/zapcore/entry.go:230 +0x565
Thu, Feb 17 2022 9:28:20 am	go.uber.org/zap.(*SugaredLogger).log(0xc000130328, 0xc00069c104, 0xfdab86, 0x23, 0xc0002d0188, 0x1, 0x1, 0x0, 0x0, 0x0)
Thu, Feb 17 2022 9:28:20 am	/go/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/sugar.go:234 +0xf6
Thu, Feb 17 2022 9:28:20 am	go.uber.org/zap.(*SugaredLogger).Panicf(...)
Thu, Feb 17 2022 9:28:20 am	/go/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/sugar.go:159
Thu, Feb 17 2022 9:28:20 am	github.com/hyperledger/fabric/common/flogging.(*FabricLogger).Panicf(...)
Thu, Feb 17 2022 9:28:20 am	/go/src/github.com/hyperledger/fabric/common/flogging/zap.go:74
Thu, Feb 17 2022 9:28:20 am	github.com/hyperledger/fabric/orderer/common/multichannel.(*Registrar).initSystemChannel(0xc000703700, 0xc0007ac040, 0x4, 0x4)
Thu, Feb 17 2022 9:28:20 am	/go/src/github.com/hyperledger/fabric/orderer/common/multichannel/registrar.go:233 +0x22e
Thu, Feb 17 2022 9:28:20 am	github.com/hyperledger/fabric/orderer/common/multichannel.(*Registrar).init(0xc000703700, 0xc0007c1890)
Thu, Feb 17 2022 9:28:20 am	/go/src/github.com/hyperledger/fabric/orderer/common/multichannel/registrar.go:163 +0xa5
Thu, Feb 17 2022 9:28:20 am	github.com/hyperledger/fabric/orderer/common/multichannel.(*Registrar).Initialize(0xc000703700, 0xc0007c1890)
Thu, Feb 17 2022 9:28:20 am	/go/src/github.com/hyperledger/fabric/orderer/common/multichannel/registrar.go:143 +0x46
Thu, Feb 17 2022 9:28:20 am	github.com/hyperledger/fabric/orderer/common/server.initializeMultichannelRegistrar(0x0, 0x0, 0xc00070aa50, 0x0, 0x0, 0xc0000c8900, 0x852, 0x853, 0xc000070c00, 0xf1, ...)
Thu, Feb 17 2022 9:28:20 am	/go/src/github.com/hyperledger/fabric/orderer/common/server/main.go:828 +0x485
Thu, Feb 17 2022 9:28:20 am	github.com/hyperledger/fabric/orderer/common/server.Main()
Thu, Feb 17 2022 9:28:20 am	/go/src/github.com/hyperledger/fabric/orderer/common/server/main.go:245 +0x10ef
Thu, Feb 17 2022 9:28:20 am	main.main()
Thu, Feb 17 2022 9:28:20 am	/go/src/github.com/hyperledger/fabric/cmd/orderer/main.go:15 +0x25

Thanks

Sure @yacovm, here's the entire stack trace:
Thu, Feb 17 2022 9:28:20 am	2022-02-17 14:28:20.681 UTC [localconfig] completeInitialization -> WARN 001 General.GenesisFile should be replaced by General.BootstrapFile
Thu, Feb 17 2022 9:28:20 am	2022-02-17 14:28:20.681 UTC [localconfig] completeInitialization -> INFO 002 Kafka.Version unset, setting to <IP_ADDRESS>
Thu, Feb 17 2022 9:28:20 am	2022-02-17 14:28:20.681 UTC [orderer.common.server] prettyPrintStruct -> INFO 003 Orderer config values:
Thu, Feb 17 2022 9:28:20 am	General.ListenAddress = "<IP_ADDRESS>"
Thu, Feb 17 2022 9:28:20 am	General.ListenPort = 7050
Thu, Feb 17 2022 9:28:20 am	General.TLS.Enabled = true
Thu, Feb 17 2022 9:28:20 am	General.TLS.PrivateKey = "/var/hyperledger/orderer/tls/server.key"
Thu, Feb 17 2022 9:28:20 am	General.TLS.Certificate = "/var/hyperledger/orderer/tls/server.crt"
Thu, Feb 17 2022 9:28:20 am	General.TLS.RootCAs = [/var/hyperledger/orderer/tls/chain.crt]
Thu, Feb 17 2022 9:28:20 am	General.TLS.ClientAuthRequired = true
Thu, Feb 17 2022 9:28:20 am	General.TLS.ClientRootCAs = [/var/hyperledger/orderer/tls/chain.crt]
Thu, Feb 17 2022 9:28:20 am	General.TLS.TLSHandshakeTimeShift = 96h0m0s
Thu, Feb 17 2022 9:28:20 am	General.Cluster.ListenAddress = ""
Thu, Feb 17 2022 9:28:20 am	General.Cluster.ListenPort = 0
Thu, Feb 17 2022 9:28:20 am	General.Cluster.ServerCertificate = ""
Thu, Feb 17 2022 9:28:20 am	General.Cluster.ServerPrivateKey = ""
Thu, Feb 17 2022 9:28:20 am	General.Cluster.ClientCertificate = "/var/hyperledger/orderer/tls/server.crt"
Thu, Feb 17 2022 9:28:20 am	General.Cluster.ClientPrivateKey = "/var/hyperledger/orderer/tls/server.key"
Thu, Feb 17 2022 9:28:20 am	General.Cluster.RootCAs = [/var/hyperledger/orderer/tls/chain.crt]
Thu, Feb 17 2022 9:28:20 am	General.Cluster.DialTimeout = 5s
Thu, Feb 17 2022 9:28:20 am	General.Cluster.RPCTimeout = 7s
Thu, Feb 17 2022 9:28:20 am	General.Cluster.ReplicationBufferSize = 20971520
Thu, Feb 17 2022 9:28:20 am	General.Cluster.ReplicationPullTimeout = 5s
Thu, Feb 17 2022 9:28:20 am	General.Cluster.ReplicationRetryTimeout = 5s
Thu, Feb 17 2022 9:28:20 am	General.Cluster.ReplicationBackgroundRefreshInterval = 5m0s
Thu, Feb 17 2022 9:28:20 am	General.Cluster.ReplicationMaxRetries = 12
Thu, Feb 17 2022 9:28:20 am	General.Cluster.SendBufferSize = 10
Thu, Feb 17 2022 9:28:20 am	General.Cluster.CertExpirationWarningThreshold = 168h0m0s
Thu, Feb 17 2022 9:28:20 am	General.Cluster.TLSHandshakeTimeShift = 96h0m0s
Thu, Feb 17 2022 9:28:20 am	General.Keepalive.ServerMinInterval = 1m0s
Thu, Feb 17 2022 9:28:20 am	General.Keepalive.ServerInterval = 2h0m0s
Thu, Feb 17 2022 9:28:20 am	General.Keepalive.ServerTimeout = 20s
Thu, Feb 17 2022 9:28:20 am	General.ConnectionTimeout = 0s
Thu, Feb 17 2022 9:28:20 am	General.GenesisMethod = "file"
Thu, Feb 17 2022 9:28:20 am	General.GenesisFile = "/var/hyperledger/orderer/genesis.block"
Thu, Feb 17 2022 9:28:20 am	General.BootstrapMethod = "file"
Thu, Feb 17 2022 9:28:20 am	General.BootstrapFile = "/var/hyperledger/orderer/genesis.block"
Thu, Feb 17 2022 9:28:20 am	General.Profile.Enabled = false
Thu, Feb 17 2022 9:28:20 am	General.Profile.Address = "<IP_ADDRESS>:6060"
Thu, Feb 17 2022 9:28:20 am	General.LocalMSPDir = "/var/hyperledger/orderer/msp"
Thu, Feb 17 2022 9:28:20 am	General.LocalMSPID = "company"
Thu, Feb 17 2022 9:28:20 am	General.BCCSP.Default = "SW"
Thu, Feb 17 2022 9:28:20 am	General.BCCSP.SW.Security = 256
Thu, Feb 17 2022 9:28:20 am	General.BCCSP.SW.Hash = "SHA2"
Thu, Feb 17 2022 9:28:20 am	General.BCCSP.SW.FileKeystore.KeyStorePath = ""
Thu, Feb 17 2022 9:28:20 am	General.Authentication.TimeWindow = 15m0s
Thu, Feb 17 2022 9:28:20 am	General.Authentication.NoExpirationChecks = true
Thu, Feb 17 2022 9:28:20 am	General.MaxRecvMsgSize = 104857600
Thu, Feb 17 2022 9:28:20 am	General.MaxSendMsgSize = 104857600
Thu, Feb 17 2022 9:28:20 am	FileLedger.Location = "/var/hyperledger/production/orderer"
Thu, Feb 17 2022 9:28:20 am	FileLedger.Prefix = ""
Thu, Feb 17 2022 9:28:20 am	Kafka.Retry.ShortInterval = 5s
Thu, Feb 17 2022 9:28:20 am	Kafka.Retry.ShortTotal = 10m0s
Thu, Feb 17 2022 9:28:20 am	Kafka.Retry.LongInterval = 5m0s
Thu, Feb 17 2022 9:28:20 am	Kafka.Retry.LongTotal = 12h0m0s
Thu, Feb 17 2022 9:28:20 am	Kafka.Retry.NetworkTimeouts.DialTimeout = 10s
Thu, Feb 17 2022 9:28:20 am	Kafka.Retry.NetworkTimeouts.ReadTimeout = 10s
Thu, Feb 17 2022 9:28:20 am	Kafka.Retry.NetworkTimeouts.WriteTimeout = 10s
Thu, Feb 17 2022 9:28:20 am	Kafka.Retry.Metadata.RetryMax = 3
Thu, Feb 17 2022 9:28:20 am	Kafka.Retry.Metadata.RetryBackoff = 250ms
Thu, Feb 17 2022 9:28:20 am	Kafka.Retry.Producer.RetryMax = 3
Thu, Feb 17 2022 9:28:20 am	Kafka.Retry.Producer.RetryBackoff = 100ms
Thu, Feb 17 2022 9:28:20 am	Kafka.Retry.Consumer.RetryBackoff = 2s
Thu, Feb 17 2022 9:28:20 am	Kafka.Verbose = false
Thu, Feb 17 2022 9:28:20 am	Kafka.Version = <IP_ADDRESS>
Thu, Feb 17 2022 9:28:20 am	Kafka.TLS.Enabled = false
Thu, Feb 17 2022 9:28:20 am	Kafka.TLS.PrivateKey = ""
Thu, Feb 17 2022 9:28:20 am	Kafka.TLS.Certificate = ""
Thu, Feb 17 2022 9:28:20 am	Kafka.TLS.RootCAs = []
Thu, Feb 17 2022 9:28:20 am	Kafka.TLS.ClientAuthRequired = false
Thu, Feb 17 2022 9:28:20 am	Kafka.TLS.ClientRootCAs = []
Thu, Feb 17 2022 9:28:20 am	Kafka.TLS.TLSHandshakeTimeShift = 0s
Thu, Feb 17 2022 9:28:20 am	Kafka.SASLPlain.Enabled = false
Thu, Feb 17 2022 9:28:20 am	Kafka.SASLPlain.User = ""
Thu, Feb 17 2022 9:28:20 am	Kafka.SASLPlain.Password = ""
Thu, Feb 17 2022 9:28:20 am	Kafka.Topic.ReplicationFactor = 3
Thu, Feb 17 2022 9:28:20 am	Debug.BroadcastTraceDir = ""
Thu, Feb 17 2022 9:28:20 am	Debug.DeliverTraceDir = ""
Thu, Feb 17 2022 9:28:20 am	Consensus = map[SnapDir:/var/hyperledger/production/orderer/etcdraft/snapshot WALDir:/var/hyperledger/production/orderer/etcdraft/wal]
Thu, Feb 17 2022 9:28:20 am	Operations.ListenAddress = ":8443"
Thu, Feb 17 2022 9:28:20 am	Operations.TLS.Enabled = false
Thu, Feb 17 2022 9:28:20 am	Operations.TLS.PrivateKey = ""
Thu, Feb 17 2022 9:28:20 am	Operations.TLS.Certificate = ""
Thu, Feb 17 2022 9:28:20 am	Operations.TLS.RootCAs = []
Thu, Feb 17 2022 9:28:20 am	Operations.TLS.ClientAuthRequired = false
Thu, Feb 17 2022 9:28:20 am	Operations.TLS.ClientRootCAs = []
Thu, Feb 17 2022 9:28:20 am	Operations.TLS.TLSHandshakeTimeShift = 96h0m0s
Thu, Feb 17 2022 9:28:20 am	Metrics.Provider = "prometheus"
Thu, Feb 17 2022 9:28:20 am	Metrics.Statsd.Network = "udp"
Thu, Feb 17 2022 9:28:20 am	Metrics.Statsd.Address = "<IP_ADDRESS>:8125"
Thu, Feb 17 2022 9:28:20 am	Metrics.Statsd.WriteInterval = 30s
Thu, Feb 17 2022 9:28:20 am	Metrics.Statsd.Prefix = ""
Thu, Feb 17 2022 9:28:20 am	ChannelParticipation.Enabled = false
Thu, Feb 17 2022 9:28:20 am	ChannelParticipation.MaxRequestBodySize = 1048576
Thu, Feb 17 2022 9:28:20 am	Admin.ListenAddress = "<IP_ADDRESS>:9443"
Thu, Feb 17 2022 9:28:20 am	Admin.TLS.Enabled = false
Thu, Feb 17 2022 9:28:20 am	Admin.TLS.PrivateKey = ""
Thu, Feb 17 2022 9:28:20 am	Admin.TLS.Certificate = ""
Thu, Feb 17 2022 9:28:20 am	Admin.TLS.RootCAs = []
Thu, Feb 17 2022 9:28:20 am	Admin.TLS.ClientAuthRequired = true
Thu, Feb 17 2022 9:28:20 am	Admin.TLS.ClientRootCAs = []
Thu, Feb 17 2022 9:28:20 am	Admin.TLS.TLSHandshakeTimeShift = 96h0m0s
Thu, Feb 17 2022 9:28:20 am	2022-02-17 14:28:20.698 UTC [orderer.common.server] initializeServerConfig -> INFO 004 Starting orderer with mutual TLS enabled
Thu, Feb 17 2022 9:28:20 am	2022-02-17 14:28:20.730 UTC [orderer.common.server] Main -> INFO 005 Not bootstrapping the system channel because of existing channels
Thu, Feb 17 2022 9:28:20 am	2022-02-17 14:28:20.800 UTC [orderer.common.server] Main -> INFO 006 Starting without a system channel
Thu, Feb 17 2022 9:28:20 am	2022-02-17 14:28:20.800 UTC [orderer.common.server] Main -> INFO 007 Setting up cluster
Thu, Feb 17 2022 9:28:20 am	2022-02-17 14:28:20.800 UTC [orderer.common.server] reuseListener -> INFO 008 Cluster listener is not configured, defaulting to use the general listener on port 7050
Thu, Feb 17 2022 9:28:20 am	2022-02-17 14:28:20.800 UTC [orderer.common.server] reuseListener -> INFO 009 Cluster listener is not configured, defaulting to use the general listener on port 7050
Thu, Feb 17 2022 9:28:20 am	2022-02-17 14:28:20.800 UTC [certmonitor] trackCertExpiration -> INFO 00a The enrollment certificate will expire on 2023-01-25 23:25:48 +0000 UTC
Thu, Feb 17 2022 9:28:20 am	2022-02-17 14:28:20.800 UTC [certmonitor] trackCertExpiration -> INFO 00b The server TLS certificate will expire on 2023-01-25 23:25:16 +0000 UTC
Thu, Feb 17 2022 9:28:20 am	2022-02-17 14:28:20.801 UTC [certmonitor] trackCertExpiration -> INFO 00c The client TLS certificate will expire on 2023-01-25 23:25:16 +0000 UTC
Thu, Feb 17 2022 9:28:20 am	2022-02-17 14:28:20.825 UTC [orderer.commmon.multichannel] initSystemChannel -> PANI 00d Error creating ledger resources: error creating channelconfig bundle: initializing channelconfig failed: could not create channel Orderer sub-group config: setting up the MSP manager failed: CA Certificate is not valid, (SN:<PHONE_NUMBER>81241201287378615618194486727802882): could not obtain certification chain: the supplied identity is not valid: x509: certificate has expired or is not yet valid: current time 2022-02-17T14:28:20Z is after 2022-02-16T16:41:01Z
Thu, Feb 17 2022 9:28:20 am	panic: Error creating ledger resources: error creating channelconfig bundle: initializing channelconfig failed: could not create channel Orderer sub-group config: setting up the MSP manager failed: CA Certificate is not valid, (SN:<PHONE_NUMBER>81241201287378615618194486727802882): could not obtain certification chain: the supplied identity is not valid: x509: certificate has expired or is not yet valid: current time 2022-02-17T14:28:20Z is after 2022-02-16T16:41:01Z
Thu, Feb 17 2022 9:28:20 am
Thu, Feb 17 2022 9:28:20 am	goroutine 1 [running]:
Thu, Feb 17 2022 9:28:20 am	go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc0002620b0, 0x0, 0x0, 0x0)
Thu, Feb 17 2022 9:28:20 am	/go/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/zapcore/entry.go:230 +0x565
Thu, Feb 17 2022 9:28:20 am	go.uber.org/zap.(*SugaredLogger).log(0xc000130328, 0xc00069c104, 0xfdab86, 0x23, 0xc0002d0188, 0x1, 0x1, 0x0, 0x0, 0x0)
Thu, Feb 17 2022 9:28:20 am	/go/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/sugar.go:234 +0xf6
Thu, Feb 17 2022 9:28:20 am	go.uber.org/zap.(*SugaredLogger).Panicf(...)
Thu, Feb 17 2022 9:28:20 am	/go/src/github.com/hyperledger/fabric/vendor/go.uber.org/zap/sugar.go:159
Thu, Feb 17 2022 9:28:20 am	github.com/hyperledger/fabric/common/flogging.(*FabricLogger).Panicf(...)
Thu, Feb 17 2022 9:28:20 am	/go/src/github.com/hyperledger/fabric/common/flogging/zap.go:74
Thu, Feb 17 2022 9:28:20 am	github.com/hyperledger/fabric/orderer/common/multichannel.(*Registrar).initSystemChannel(0xc000703700, 0xc0007ac040, 0x4, 0x4)
Thu, Feb 17 2022 9:28:20 am	/go/src/github.com/hyperledger/fabric/orderer/common/multichannel/registrar.go:233 +0x22e
Thu, Feb 17 2022 9:28:20 am	github.com/hyperledger/fabric/orderer/common/multichannel.(*Registrar).init(0xc000703700, 0xc0007c1890)
Thu, Feb 17 2022 9:28:20 am	/go/src/github.com/hyperledger/fabric/orderer/common/multichannel/registrar.go:163 +0xa5
Thu, Feb 17 2022 9:28:20 am	github.com/hyperledger/fabric/orderer/common/multichannel.(*Registrar).Initialize(0xc000703700, 0xc0007c1890)
Thu, Feb 17 2022 9:28:20 am	/go/src/github.com/hyperledger/fabric/orderer/common/multichannel/registrar.go:143 +0x46
Thu, Feb 17 2022 9:28:20 am	github.com/hyperledger/fabric/orderer/common/server.initializeMultichannelRegistrar(0x0, 0x0, 0xc00070aa50, 0x0, 0x0, 0xc0000c8900, 0x852, 0x853, 0xc000070c00, 0xf1, ...)
Thu, Feb 17 2022 9:28:20 am	/go/src/github.com/hyperledger/fabric/orderer/common/server/main.go:828 +0x485
Thu, Feb 17 2022 9:28:20 am	github.com/hyperledger/fabric/orderer/common/server.Main()
Thu, Feb 17 2022 9:28:20 am	/go/src/github.com/hyperledger/fabric/orderer/common/server/main.go:245 +0x10ef
Thu, Feb 17 2022 9:28:20 am	main.main()
Thu, Feb 17 2022 9:28:20 am	/go/src/github.com/hyperledger/fabric/cmd/orderer/main.go:15 +0x25

@yacovm Yes, I think we need a timeshift for regular CA and TLS CA verification. Instead of all of these different timeshift options, what would you think of a single global timeshift that would apply to all of them? Users have a difficult time keeping track of all the different certs and which overrides apply to which ones.

@yacovm Yes, I think we need a timeshift for regular CA and TLS CA verification. Instead of all of these different timeshift options, what would you think of a single global timeshift that would apply to all of them? Users have a difficult time keeping track of all the different certs and which overrides apply to which ones.

Please click on the links in my comment and read the code and understand why a "global time shift" or TLSHANDSHAKETIMESHIFT is irrelevant to the problem at hand.

Ah, when you said "use the time override" I thought you were talking about the timeshift override.
Rather, you are suggesting to override the check time based on the CA cert NotBefore time so that it ALWAYS passes when validating CA and TLS CA certs.
Are you suggesting this be hardcoded to always pass (like the identity checks that you referenced), or based on a new orderer/peer config option like NoExpirationChecks?

Are you suggesting this be hardcoded to always pass (like the identity checks that you referenced), or based on a new orderer/peer config option like NoCAExpirationChecks?

The former, of course. The no expiration checks is a check for validation in the admission path of transactions.
Here we're dealing with code path that initializes the channel config bundle.

Ok, I agree with the proposal.
To take an extreme example... say you shut down a peer for 30 years and then start it back up... it shouldn't fail to start simply because the old CA certs in the channel config have expired. It should be allowed to start and process blocks so that it can catch up to the latest channel config containing the latest CA certs.  (Of course, you will likely need to update the peer's own local certs, but that is a different story and under control of the peer admin).

Hi All,
Just a comment here to make sure everything is in context.
When a peer restarts, the peer should rescan the ledger and apply the configuration blocks in the order given by the ledger. It is clear that, the peer should not panic because a configuration block in the middle contains expired certificates because that block refers to a previous time period. For this reason, I agree that we must fix this and align the validation of TLS CA certs to what we do for the others CA certs.
Having said that, it must be clear that the administrators of the Fabric network must ensure that all CA certificates used in the current epoch are not expired. Fabric cannot check that, it must be done manually or with other automated processes.

Having said that, it must be clear that the administrators of the Fabric network must ensure that all CA certificates used in the current epoch are not expired. Fabric cannot check that, it must be done manually or with other automated processes.

But the TLS CA certificates are not going to be used by the channel config at all.
If anything, the expired ones will not be used to validate certificates at all, because the only place that uses them is the TLS configuration which does take into account their expiration.

Fix backported to release-2.4, release-2.3, release-2.2.

It also affects Fabric 1.4.x ? Looks like the shift gets nor applied on the for the root CA.

@rodolfoleal Fabric v1.4 is out of maintenance. There are many issues with v1.4 that have been fixed only in v2.x. Users must upgrade to v2.x to get these fixes.


Error loading Font List

New Update create error on each font Windows 1909 (Build18363.657)

@Avidin Thanks, and sorry about this! Submitted a pull request that fixes this.
(Also slightly irks me that I replaced the error dialog everywhere in the app with one that provides error details for GitHub, apart from this one place 😅)

Version 2020.5.2 is released to store to fix this issue


fix the string validator for rack names

I discovered this limit while writing configs for the AI racks.

I think that "between" might need to be 0-18.
Warrants an experiment at least.

│ Error: Invalid Attribute Value Length
│
│   with apstra_rack_type.AI_16xA100,
│   on racks.tf line 30, in resource "apstra_rack_type" "AI_16xA100":
│   30:   name                       = "012345678901234567"
│
│ Attribute name string length must be between 1 and 17, got: 18
Admittedly confusing, but works :-)


Publish a 0.9 version

In the last ~week multiple changes have made it in. We should publish a new version with these changes. No APIs were broken (as far as I know) so bumping to 0.8.1 or 0.9 seem appropriate.

Hello @bramp,
On the next version please update to handle "N/A" Time like this:

I think from the following code.

Thanks bro...

I think this was fixed in https://github.com/bramp/ffmpeg-cli-wrapper/issues/302

@bramp The same user also commented on the given issue a week ago. So I think it's okay to assume the fix he needs is included

Now I am using this library using Maven, but the latest version I use is 0.8.0... how do I use it for version 0.8.1 (which has fixed the "N/A" error handling issue), thanks bro.

There is no 0.8.1, wait for this bug to be resolved when we release the next version. If you can't wait, you can check the project out, and build it yourself for maven to use.

@bramp
Tell me, when do you plan to release a new version?

+1 for a new version!

+1
Need threads option ASAP!

+1 new version please, I need pr for id chapter to long instead int

Any update on when a new version will be released? @bramp

Hello @bramp You can upgrade version of logback (1.5.6) that is detected to be a vulnerability. Thanks you so much

@bramp I think it's high time for 0.9.0. What do you think?

so is there any eta?


[Feature Request]: Multiple value filters in Qdrant Vectorstore

Feature Description
Qdrant Vector store Metadata filters
Allow users to have multiple values for the same key
eg : {'key':'country','values':{'usa','uk'}
Qdrant has already implemented this in their python client, allows adding MatchAny to the filters. LlamaIndex currently supports only single value for a key
https://qdrant.tech/documentation/concepts/filtering/#match-any
Reason
No response
Value of Feature
No response

Thanks for this suggestion @AravAct! Seems like a worthwhile addition to our library.

Hello, I would like to check if I understand the requirement.
In \vector_stores\qdrant.py  I added to the  _build_query_filter() this extra conditional for check if the value is a list. If so, wrap those values in a MatchAny condition:
if isinstance(subfilter.value, list):
                    must_conditions.append(
                        FieldCondition(
                            key=subfilter.key,
                            match=MatchAny(any=subfilter.value),
                        )
                    )

Leaving the function as follows:
def _build_query_filter(self, query: VectorStoreQuery) -> Optional[Any]:
        if not query.doc_ids and not query.query_str:
            return None

        from qdrant_client.http.models import (
            FieldCondition,
            Filter,
            MatchAny,
            MatchValue,
            Range,
        )

        must_conditions = []

        if query.doc_ids:
            # Using MatchAny for filtering by doc_ids
            must_conditions.append(
                FieldCondition(
                    key="doc_id",
                    match=MatchAny(any=query.doc_ids),
                )
            )

        if query.node_ids:
            # Using MatchAny for filtering by node_ids
            must_conditions.append(
                FieldCondition(
                    key="id",
                    match=MatchAny(any=query.node_ids),
                )
            )

        # Qdrant does not use the query.query_str property for the filtering. Full-text
        # filtering cannot handle longer queries and can effectively filter our all the
        # nodes. See: https://github.com/jerryjliu/llama_index/pull/1181

        if query.filters is None:
            return Filter(must=must_conditions)

        for subfilter in query.filters.filters:
            if isinstance(subfilter.value, float):
                must_conditions.append(
                    FieldCondition(
                        key=subfilter.key,
                        range=Range(
                            gte=subfilter.value,
                            lte=subfilter.value,
                        ),
                    )
                )
            else:
                # Using MatchAny if subkey has multiple values
                if isinstance(subfilter.value, list):
                    must_conditions.append(
                        FieldCondition(
                            key=subfilter.key,
                            match=MatchAny(any=subfilter.value),
                        )
                    )
                else:
                    must_conditions.append(
                        FieldCondition(
                            key=subfilter.key,
                            match=MatchValue(value=subfilter.value),
                        )
                    )

        return Filter(must=must_conditions)

Is this the correct way to implement this feature?
I am open to any feedback. Thanks.

@AravAct we are refactoring our MetadataFilter here https://github.com/run-llama/llama_index/pull/9216
Feel free to add more customized filter for qdrant vectorstore. you can follow what we did for Chroma


MSSQL Warnings cause "Previous SQL Not a Query" Errors

Environment
To diagnose, we usually need to know the following, including version numbers.  On Windows, be
sure to specify 32-bit Python or 64-bit:

Python: 2.7 32-bit
pyodbc: 4.0.16
OS: Windows 10
DB: MSSQL (Mgmt studio 17.9.1, not sure of actual DB version)
driver: unsure

Issue
If I run a query that generates warnings, pyodbc will fail, claiming the SQL is not a query. I contend that MSSQL warnings should not nullify your query execution in pyodbc.  I'd expect that I could at least set a flag on the query to ignore warnings.
I've trolled through the internet for several years hoping for a resolution to this but have never found a satisfactory one.  I eliminate warnings in my SQL whenever I can, but not all the entities I use are under my control, and some of those are (I am assured by their authors) not able to suppress the warnings.
(This is true primarily with some null aggregation warnings that occur in a view that performs a pivot.)
Currently I toggle ansi_warnings off and on around the offending code, but that solution doesn't support cross-server queries which can be problematic when shifting from UAT to production environments.  It also causes some entities to fail with the followwing eror:
Heterogeneous queries require the ANSI_NULLS and ANSI_WARNINGS options to be set for the connection. This ensures consistent query semantics. Enable these options and then reissue your query.
My hope is that this might also resolve the problem where SET NOCOUNT ON is required.
Thanks for all your fine work!

Can you give an example of such a query and DB schema?

Please also post an ODBC trace

Will do.  Working on a good example now - it seems my trivial one doesn't
reproduce so there may be additional complexities here that I'm trying to
track down.
Not sure if I'll be able to get an ODBC trace for you since I'm not an
admin but I'll speak w/ our DBAs and see what I can do.
On Tue, Feb 26, 2019 at 9:49 AM v-chojas<EMAIL_ADDRESS>wrote:

Please also post an ODBC trace
https://docs.microsoft.com/en-us/sql/odbc/admin/setting-tracing-options
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/mkleehammer/pyodbc/issues/534#issuecomment-467492297,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AtzEqDxTFA-PyoLrz1WasdijLoi6kbgRks5vRVedgaJpZM4bSYhi
.

The exercise of putting together a naive reproduction of this has led me to believe that either the problem no longer exists or we mis-diagnosed it in the first place.
The errors I was encountering today were actually due to the heterogeneous query error caused by my "set ansi_warnings off" on the cross-server query, which I did because I've always been convinced that null aggregate warnings will break my script.
I'll close this issue out.  Thanks a lot for your quick response and sorry for taking up your time!


Syncplay dependancy issue

Describe the bug
syncplay is not work currently working on Debian while I have not tried a fresh install i.e. vm following the build instructions yields dependency issues
To Reproduce
Steps to reproduce the behavior:
follow
Expected behavior
syncplay gui should start
output
running syncplay using
$syncplay -g
yeilds:
syncplay -g /usr/lib/syncplay/syncplay/utils.py:304: SyntaxWarning: invalid escape sequence '\g'   return re.sub(constants.ROOM_NAME_STRIP_REGEX, "\g<roomnamebase>", RoomName) /usr/lib/syncplay/syncplay/utils.py:482: SyntaxWarning: invalid escape sequence '\+'   CONTROLLED_ROOM_REGEX = re.compile("^\+(.*):(\w{12})$") /usr/lib/syncplay/syncplay/utils.py:483: SyntaxWarning: invalid escape sequence '\d'   PASSWORD_REGEX = re.compile("[A-Z]{2}-\d{3}-\d{3}") /usr/lib/syncplay/syncplay/constants.py:115: SyntaxWarning: invalid escape sequence '\.'   FILENAME_STRIP_REGEX = "[-~_\.\[\](): ]" /usr/lib/syncplay/syncplay/constants.py:116: SyntaxWarning: invalid escape sequence '\-'   CONTROL_PASSWORD_STRIP_REGEX = "[^a-zA-Z0-9\-]" /usr/lib/syncplay/syncplay/constants.py:117: SyntaxWarning: invalid escape sequence '\+'   ROOM_NAME_STRIP_REGEX = "^(\+)(?P<roomnamebase>.*)(:)(\w{12})$" /usr/lib/syncplay/syncplay/constants.py:165: SyntaxWarning: invalid escape sequence '\m'   MPC64_EXECUTABLES = ["mpc-hc64.exe", "mpc-hc64_nvo.exe", "x64\mpc-hc\shoukaku.exe"] /usr/lib/syncplay/syncplay/constants.py:303: SyntaxWarning: invalid escape sequence '\.'   MPLAYER_ANSWER_REGEX = "^ANS_([a-zA-Z_-]+)=(.+)$|^(Exiting)\.\.\. \((.+)\)$" Could not import GUI libraries. If you do not have PySide installed then you will need to install it for the GUI to work. Hostname can't be empty! Some necessary arguments are missing, refer to --help
I think this is an issue with python version (i have messed around with older version) so i run from the release tarball
$ python3.12 syncplayClient.py
python3.12 syncplayClient.py -g Could not import GUI libraries. If you do not have PySide installed then you will need to install it for the GUI to work. Hostname can't be empty! Some necessary arguments are missing, refer to --help
Version and platform:
-Debian 12
side note I do not know if this is a me issue however I have also run from a venv and received the same result I would recommend making a flatpak of the app to fix future dependency issues.

I'm not a Linux user so will mostly rely on others to provide support on this, but I do have some questions that will hopefully move things forwards:

What version of Syncplay and PySide are you using?
Does Syncplay run okay if you run it in --no-gui mode?

side note I do not know if this is a me issue however I have also run from a venv and received the same result I would recommend making a flatpak of the app to fix future dependency issues.

According to https://github.com/Syncplay/syncplay/issues/199 Flatpak "seems to be incompatible with current model of integration with media players Syncplay uses".
As per https://github.com/Syncplay/syncplay/issues/643 we tried AppImages and Snap and it ended up causing more compatibility issues than it solved. That issue does however include a potential option in terms of using pipx but nobody has tried it yet to see if it works.

@ChunkyPanda03 All of the problems you described aside - Syncplay is just in the Debian repositories. So unless you really need the latest version, you can just # apt install syncplay.
I'll second the question of @Et0h what version of pyside do you have and how it is installed? The message you are getting is basically a simple wrapper on failed import of pyside.
Regarding flatpak, as well as snaps and appimages - between Syncplay being Python application that also uses Qt and the need to control external media player, it turned out in our experience that packaging it into any of those is very difficult. For flatpak we never figured where to even start and AppImage as well as Snap that we managed to build at some point were either unreliable or straight up broken.

@daniel-123  running it through the downloaded deb apt repo gets me this however I know my current set up is messy and slightly unorthodox so I don't know if this is a me issue.
@Et0h Syncplay was running whatever the latest "release" build was on the git repo and what command should I run to find the pyside version I know there are deb packages and pip installs. And on the terms of flatpak and other runners is there any provided venv configurations you all use? Also it does work fine in "--no-gui" mode I think it is a pyside issue.
running the old apt image works fine however.
Attribute Qt::AA_EnableHighDpiScaling must be set before QCoreApplication is created.
Traceback (most recent call last):
  File "/usr/bin/syncplay", line 33, in <module>
    sys.exit(load_entry_point('syncplay==1.6.9', 'gui_scripts', 'syncplay')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/syncplay/ep_client.py", line 8, in main
    SyncplayClientManager().run()
  File "/usr/lib/python3/dist-packages/syncplay/clientManager.py", line 9, in run
    config = ConfigurationGetter().getConfiguration()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/syncplay/ui/ConfigurationGetter.py", line 544, in getConfiguration
    self._forceGuiPrompt()
  File "/usr/lib/python3/dist-packages/syncplay/ui/ConfigurationGetter.py", line 455, in _forceGuiPrompt
    for key, value in list(self._promptForMissingArguments().items()):
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/syncplay/ui/ConfigurationGetter.py", line 420, in _promptForMissingArguments
    gc.run()
  File "/usr/lib/python3/dist-packages/syncplay/ui/GuiConfiguration.py", line 40, in run
    dialog = ConfigDialog(self.config, self._availablePlayerPaths, self.error, self.defaultConfig)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/syncplay/ui/GuiConfiguration.py", line 1420, in __init__
    self.setWindowFlags(self.windowFlags() & Qt.WindowCloseButtonHint & ~Qt.WindowContextHelpButtonHint)
TypeError: 'PySide2.QtCore.Qt.WindowFlags' object cannot be interpreted as an integer

I have now updated Syncplay to hopefully provide more detail in the error messages when loading PySide fails. See GIT HEAD / https://github.com/Syncplay/syncplay/actions/runs/7978410389
My understanding is that if you are using Python >=3.12 then you will need to have PySide6 installed as earlier versions of PySide only support up to 3.11.
A quick look on packages.debian.org seems to indicate that there is no PySide6 packages for Debian yet. I assume this means you would either need to install PySide6 via pip install pyside6 or equivalent or move to Python 3.11 so you can use sudo apt install python3-pyside2.qtgui, but I don't use Debian so I can't say for sure.

ok I have it,
I first off purged and fixed some of my python packages and installs I think that my installed old python 3.4 build was messing with me.
then after some on and off tinkering I have a method for getting it to run

when making and installing somehow the internal permissions of the install location were incorrect. found this when it was claiming that certain files didn't exist. fixed it by nuking it with a sudo chmod 777 -R ... on the affected folders. I will look into this more later to see if this is my fault.

the program started working but with no gui fixed this by pip install pyside6 then it complained that qt.qpa.plugin: From 6.5.0, xcb-cursor0 or libxcb-cursor0 is needed to load the Qt xcb platform plugin. this was fixed by 'sudo apt install libxcb-cursor0'

tada syncplay -g now works

going to keep this open to test the odd folder permission issue will close later

If you use flatpak vlc you can add the extension by creating a directory $HOME/.var/app/org.videolan.VLC/data/vlc/lua/intf/ and copying syncplay.lua in it


Bump version to 2.7.0

Hi
The latest version of this package is not usable with TypeScript due to #280, can you release a new version so we can use this package in our project... : )
thanks

a8e73dbef54e9bca91eeac3cdbbb8217da7168d2


Invalid token

Looking over the code I saw it to be standard asyncio discord.py, I made a bot user and it gives me "discord.errors.LoginFailure: Improper token has been passed". Where do I get this token from if not a bot user?

Hi,
You should look here :
Https://github.com/Tyrrrz/DiscordChatExporter/wiki/Obtaining-Token-and-Channel-IDs
or
https://greasyfork.org/en/scripts/395741-get-discord-token

Thanks, I figured it out, any chance you could include that inside the readme?

It's added! 920df8a890ed8ab8e38ccae6daf62f1a69b8364f


Fix ci_gen_kustomize_values role name in files

Fix the ci_gen_kustomize_values role name in the files section for the
downstream hci va job, which was missing the _values suffix.

/lgtm

/approve