Assert failed after successfully import corpus generated by a JS2FuzzIL Compiler

Hello Saelo!
I wrote a tool that can convert JS code into FuzzIL intermediate language. After collecting seeds from the network, more than 400 corpus were successfully generated and imported into Fuzzilli for Fuzzing. The command line is like this
swift run FuzzilliCli --profile=jsc /media/hdd/WebKit-git/FuzzBuild_96c26fd_20210427_partial_instru/Debug/bin/jsc --storagePath=fuzz_tmp --overwrite --engine=mutation --exportStatistics- -diagnostics --timeout=500 --jobs=1 --importCorpusAll=JSCorpus_fuzz_protobuf_myCompiler
But after corpus is successfully imported and fuzz the target js engine for several hundred times, Fuzzilli crashed, Fuzzilli/Program.swift:55: Assertion failed
Fuzzilli crashes  in the following code
/// Constructs a program with the given code. The code must be statically valid.
    public init(with code: Code) {
        assert(code.isStaticallyValid())
        self.code = code
    }

After a simple debugging, you can see that Fuzzilli selected a corpus to mutate after successfully importing corpus. When checking the new code generated by the mutation, assert failed. What is confusing is that when Fuzzilli is importing corpus, code.isStaticallyValid will also be called to check the corpus. If the check fails, the corpus will be discarded. Therefore, the imported corpus must have passed the check. Why is it that after only a few hundred runs, a corpus that is not statically valid is generated?
One possible solution that I currently think of is to make changes to all the assert checks in the project. If the checks fail, the mutated seeds are directly discarded and a new round of mutation is performed. I wonder if there is a better solution.
In addition, I also tried the Compiler written by ocaml that comes with the project, and there is a similar problem. After importing corpus and running for a period of time, Fuzzilli itself will crash (depending on the corpus, it may only be executed dozens to a million times). But the point of the crash is different. The crash is in the setType function of Sources/Fuzzilli/FuzzIL/AbstractInterpreter.swift
    public mutating func setType(of v: Variable, to t: Type) {
        // Variables must not be .anything or .nothing. For variables that can be anything, .unknown is the correct type.
        assert(t != .anything && t != .nothing)
        state.updateType(of: v, to: t)
    }

I tried adding --noAbstractInterpretation and it won't crash. So is the inspection of corpus not thorough enough when importing corpus? Sincerely hope that you could  provide some guidance.
The complete backtrace when the program crashes is as follows:
* thread #3, name ='FuzzilliCli', stop reason = Assertion failed
    frame #0: 0x00007ffff7dff1e0 libswiftCore.so`_swift_runtime_on_report
    frame #1: 0x00007ffff7e74092 libswiftCore.so`_swift_stdlib_reportFatalErrorInFile + 210
    frame #2: 0x00007ffff7b5ce95 libswiftCore.so`Swift._assertionFailure(_: Swift.StaticString, _: Swift.String, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 597
    frame #3: 0x00005555556eb0cd FuzzilliCli`Program.init(code=Fuzzilli.Code @ 0x00007fffebffdde0, self=0x00007fffe401b7f0) at Program.swift:55:9
    frame #4: 0x00005555556eafa9 FuzzilliCli`Program.__allocating_init(with:) at Program.swift:0
    frame #5: 0x00005555556eb1c1 FuzzilliCli`Program.__allocating_init(code=Fuzzilli.Code @ 0x00007fffebffdf18, parent=nil, types=Fuzzilli.ProgramTypes @ 0x00007fffebffdf08, comments=Fuzzilli.ProgramComments @ 0x00007fffebswift: at Program.14ffwift)
  * frame #6: 0x00005555556293ea FuzzilliCli`ProgramBuilder.finalize(self=0x00007fffe4018fc0) at ProgramBuilder.swift:99:23
    frame #7: 0x00005555557a8455 FuzzilliCli`BaseInstructionMutator.mutate(program=0x00007fffe4003840, b=0x00007fffe4018fc0, self=0x000055555606bdc0) at BaseInstructionMutator.swift:51:18
    frame #8: 0x00005555557b2271 FuzzilliCli`Mutator.mutate(program=0x00007fffe4003840, fuzzer=0x00005555560828e0, self=0x000055555606bdc0) at Mutator.swift:28:16
    frame #9: 0x0000555555623e60 FuzzilliCli`MutationEngine.fuzzOne(group=0x00005555560a9800, self=0x00005555560796c0) at MutationEngine.swift:129:41
    frame #10: 0x0000555555624d91 FuzzilliCli`protocol witness for FuzzEngine.fuzzOne(_:) in conformance MutationEngine at <compiler-generated>:0
    frame #11: 0x000055555572412d FuzzilliCli`Fuzzer.fuzzOne(self=0x00005555560828e0) at Fuzzer.swift:624:16
    frame #12: 0x00005555557242f6 FuzzilliCli`closure #1 in Fuzzer.fuzzOne(self=0x00005555560828e0) at Fuzzer.swift:629:18
    frame #13: 0x0000555555653930 FuzzilliCli`thunk for @escaping @callee_guaranteed () -> () at <compiler-generated>:0
    frame #14: 0x00007ffff7006947 libdispatch.so`_dispatch_call_block_and_release + 7
    frame #15: 0x00007ffff7012259 libdispatch.so`_dispatch_lane_serial_drain + 729
    frame #16: 0x00007ffff7012efe libdispatch.so`_dispatch_lane_invoke + 878
    frame #17: 0x00007ffff701ab72 libdispatch.so`_dispatch_worker_thread + 514
    frame #18: 0x00007ffff71dc609 libpthread.so.0`start_thread(arg=<unavailable>) at pthread_create.c:477:8
    frame #19: 0x00007ffff6f02293 libc.so.6`__clone at clone.S:95

The complete output information of the program is as follows:
[Cli] Deleting all files in /home/dddong/workspace/fuzzilli/fuzz_tmp due to --overwrite
[Coverage] Initialized, 66032 edges
[JavaScriptEnvironment] initialized static JS environment model
[JavaScriptEnvironment] Have 63 available builtins: ["RegExp", "createGlobalObject", "Int8Array", "Symbol", "SyntaxError", "forceGCSlowPaths", "noInline", "transferArrayBuffer", "placeholder", "Uint8Array", "Infinity", "arguments", "NaN", "Boolean", "undefined", "AggregateError", "Uint16Array", "parseInt", "Float64Array", "Int16Array", "parseFloat", "drainMicrotasks", "Promise", "Math", "RangeError", "this", "ArrayBuffer", "WeakMap", "gc", "OSRExit", "WeakSet", "Set", "ensureArrayStorage", "ReferenceError", "Map", "fiatInt52", "noFTL", "fullGC", "eval", "print", "BigInt", "String", "Function", "DataView", "Proxy", "makeMasquerader", "Int32Array", "Object", "Number", "Uint8ClampedArray", "Uint32Array", "JSON", "Array", "Reflect", "Date", "edenGC", "TypeError", "runString", "isNaN", "EvalError", "Float32Array", "isFinite", "Error"]
[JavaScriptEnvironment] Have 211 available method names: ["setUTCHours", "getUTCMonth", "getFloat64", "getUTCFullYear", "getMinutes", "setDate", "getOwnPropertyNames", "getSeconds", "setFullYear", "setUint32", "repeat", "getUTCMinutes", "sinh", "match", "defineProperty", "isNaN", "getOwnPropertyDescriptors", "getFullYear", "getTime", "freeze", "trim", "getPrototypeOf", "fromEntries", "find", "join", "assign", "m", "getOwnPropertyDescriptor", "setUTCDate", "deleteProperty", "compile", "toGMTString", "getUTCMilliseconds", "getFloat32", "setMilliseconds", "all", "substring", "o", "setInt8", "from", "setFloat64", "isSafeInteger", "lastIndexOf", "defineProperties", "setSeconds", "sort", "subarray", "findIndex", "p", "flatMap", "push", "for", "isExtensible", "acos", "indexOf", "pow", "getTimezoneOffset", "flat", "trimRight", "race", "getMilliseconds", "max", "cos", "clear", "forEach", "getOwnPropertySymbols", "setHours", "getUTCDate", "keys", "values", "slice", "atan", "floor", "padEnd", "asUintN", "apply", "concat", "asinh", "map", "isFrozen", "getUint32", "reduceRight", "atanh", "reject", "padStart", "every", "isSealed", "setMinutes", "filter", "delete", "get", "log", "log1p", "getInt32", "atan2", "setUTCMinutes", "tan", "copyWithin", "raw", "search", "fromCodePoint", "random", "isFinite", "round", "seal", "asin", "fround", "now", "reverse", "trimLeft", "getInt8", "log10", "call", "setUint16", "sign", "replace", "expm1", "toISOString", "toUpperCase", "resolve", "reduce", "codePointAt", "create", "getUTCDay", "toJSON", "n", "toTimeString", "test", "imul", "charCodeAt", "toString", "add", "ownKeys", "asIntN", "ceil", "clz32", "setMonth", "finally", "cbrt", "cosh", "trimEnd", "trunc", "replaceAll", "then", "exec", "of", "fromCharCode", "getHours", "min", "getInt16", "UTC", "exp", "set", "splice", "getUint8", "getDay", "setPrototypeOf", "catch", "getUTCHours", "toLowerCase", "setYear", "includes", "getDate", "construct", "shift", "toUTCString", "endsWith", "sqrt", "isArray", "setUTCMonth", "is", "log2", "trimStart", "getYear", "bind", "setInt32", "setUTCSeconds", "setUint8", "some", "split", "toLocaleString", "setUTCFullYear", "abs", "fill", "unshift", "getMonth", "setTime", "setUTCMilliseconds", "hypot", "localeCompare", "setInt16", "stringify", "pop", "getUTCSeconds", "preventExtensions", "allSettled", "acosh", "sin", "startsWith", "isView", "keyFor", "matchAll", "charAt", "isInteger", "tanh", "getUint16", "setFloat32", "has", "parse", "toDateString", "entries"]
[JavaScriptEnvironment] Have 51 property names that are available for read access: ["toStringTag", "source", "multiline", "message", "description", "match", "flags", "c", "global", "toString", "sticky", "MIN_VALUE", "search", "asyncIterator", "d", "buffer", "constructor", "E", "dotAll", "byteOffset", "arguments", "iterator", "length", "hasInstance", "b", "matchAll", "prototype", "e", "NaN", "name", "unicode", "valueOf", "POSITIVE_INFINITY", "MAX_VALUE", "replace", "unscopable", "ignoreCase", "isConcatSpreadable", "toPrimitive", "MAX_SAFE_INTEGER", "species", "PI", "__proto__", "split", "byteLength", "NEGATIVE_INFINITY", "a", "caller", "size", "MIN_SAFE_INTEGER", "EPSILON"]
[JavaScriptEnvironment] Have 10 property names that are available for write access: ["a", "toString", "c", "__proto__", "valueOf", "length", "e", "d", "constructor", "b"]
[JavaScriptEnvironment] Have 5 custom property names: ["d", "b", "a", "c", "e"]
[JavaScriptEnvironment] Have 4 custom method names: ["p", "m", "n", "o"]
[Fuzzer] Initialized
Process 2831786 stopped and restarted: thread 1 received signal: SIGCHLD
Process 2831786 stopped and restarted: thread 1 received signal: SIGCHLD
Process 2831786 stopped and restarted: thread 1 received signal: SIGCHLD
[Fuzzer] Recommended timeout: at least 100ms. Current timeout: 500ms
[Fuzzer] Startup tests finished successfully
[Cli] Starting All-corpus import of 470 programs.  Corpus now contains 455 elements
[Fuzzer] Let's go!
Fuzzer Statistics
-----------------
Total Samples:                10
Interesting Samples Found:    454
Valid Samples Found:          9
Corpus Size:                  455
Correctness Rate:             90.00%
Timeout Rate:                 0.00%
Crashes Found:                0
Diff Found:                   0
Timeouts Hit:                 0
Coverage:                     11.12%
Avg. program size:            50.40
Connected workers:            0
Execs / Second:               8.05
Fuzzer Overhead:              53.87%
Total Execs:                  501

[JavaScriptEnvironment] Missing type for builtin WebAssembly
[JavaScriptEnvironment] Missing type for builtin kSig_v_v
[JavaScriptEnvironment] Missing type for builtin kWasmAnyFunc
[JavaScriptEnvironment] Missing type for builtin WasmModuleBuilder
[JavaScriptEnvironment] Missing type for builtin WebAssembly
[JavaScriptEnvironment] Missing type for builtin kSig_v_v
[JavaScriptEnvironment] Missing type for builtin kWasmAnyFunc
[JavaScriptEnvironment] Missing type for builtin Intl
[JavaScriptEnvironment] Missing type for builtin document
[JavaScriptEnvironment] Missing type for builtin WasmModuleBuilder
[JavaScriptEnvironment] Missing type for builtin kSig_v_v
[JavaScriptEnvironment] Missing type for builtin WasmModuleBuilder
Fuzzilli/Program.swift:55: Assertion failed
Current stack trace:
0    libswiftCore.so                    0x00007ffff7dff210 swift_reportError + 50
1    libswiftCore.so                    0x00007ffff7e73fc0 _swift_stdlib_reportFatalErrorInFile + 112
2    libswiftCore.so                    0x00007ffff7b5cc40 _assertionFailure(_:_:file:line:flags:) + 597
3    FuzzilliCli                        0x00005555556eb0cd <unavailable> + 1667277
4    FuzzilliCli                        0x00005555556eafa9 <unavailable> + 1666985
5    FuzzilliCli                        0x00005555556eb1c1 <unavailable> + 1667521
6    FuzzilliCli                        0x00005555556293ea <unavailable> + 873450
7    FuzzilliCli                        0x00005555557a8455 <unavailable> + 2442325
8    FuzzilliCli                        0x00005555557b2271 <unavailable> + 2482801
9    FuzzilliCli                        0x0000555555623e60 <unavailable> + 851552
10   FuzzilliCli                        0x0000555555624d91 <unavailable> + 855441
11   FuzzilliCli                        0x000055555572412d <unavailable> + 1900845
12   FuzzilliCli                        0x00005555557242f6 <unavailable> + 1901302
13   FuzzilliCli                        0x0000555555653930 <unavailable> + 1046832
14   libdispatch.so                     0x00007ffff7006947 <unavailable> + 141639
15   libdispatch.so                     0x00007ffff7012259 <unavailable> + 189017
16   libdispatch.so                     0x00007ffff7012efe <unavailable> + 192254
17   libdispatch.so                     0x00007ffff701ab72 <unavailable> + 224114
18   libpthread.so.0                    0x00007ffff71dc609 <unavailable> + 38409
19   libc.so.6                          0x00007ffff6f02250 clone + 67

Hi! Thanks for the detailed report!
What you describe does indeed sound like a bug in Fuzzilli, i.e. successfully imported samples should never become statically invalid due to mutations, and the importer should never allow invalid samples to be imported in the first place.
Are you able to reproduce the issue? I think it would help if you could change the Program.init method to something like:
    public init(with code: Code) {
        do {
            try code.check()
        } catch FuzzilliError.codeVerificationError(let msg) {
            print("Invalid program found: \(msg)")
            print("Code:\n\(FuzzILLifter().lift(code))")
            abort()
        } catch {
            print("??? \(error)")
        }
        //assert(code.isStaticallyValid())
        self.code = code
    }

Maybe you can also put a print("Mutating with \(mutator.name)") here to see which mutator caused the invalid program. Finally, you could hack the --inspect=history feature to e.g. print everything to stdout. That should give you a fairly detailed log of what happened.
The other issue you are describing with the existing ocaml compiler sounds like it might be issue https://github.com/googleprojectzero/fuzzilli/issues/199 so that should be unrelated.

Very grateful for such a prompt reply. Yes, I can reproduce this issue, I
will try your method to see where the bug is. 😀
Samuel Groß @.***> 于2021年6月29日周二 下午5:14写道：

Hi! Thanks for the detailed report!
What you describe does indeed sound like a bug in Fuzzilli, i.e.
successfully imported samples should never become statically invalid due to
mutations, and the importer should never allow invalid samples to be
imported in the first place.
Are you able to reproduce the issue? I think it would help if you could
change the Program.init method to something like:
public init(with code: Code) {
    do {
        try code.check()
    } catch FuzzilliError.codeVerificationError(let msg) {
        print("Invalid program found: \(msg)")
        print("Code:\n\(FuzzILLifter().lift(code))")
        abort()
    } catch {
        print("??? \(error)")
    }
    //assert(code.isStaticallyValid())        self.code = code
}

Maybe you can also put a print("Mutating with (mutator.name)") here
https://github.com/googleprojectzero/fuzzilli/blob/f31ba8ee5f1d7d5f338a0d047a55d21b13494f7a/Sources/Fuzzilli/Core/MutationEngine.swift#L128
to see which mutator caused the invalid program. Finally, you could hack
the --inspect=history
https://github.com/googleprojectzero/fuzzilli/blob/f31ba8ee5f1d7d5f338a0d047a55d21b13494f7a/Sources/FuzzilliCli/main.swift#L85
feature to e.g. print everything to stdout. That should give you a fairly
detailed log of what happened.
The other issue you are describing with the existing ocaml compiler sounds
like it might be issue #199
https://github.com/googleprojectzero/fuzzilli/issues/199 so that should
be unrelated.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/googleprojectzero/fuzzilli/issues/218#issuecomment-870424596,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQB67X5NWTLGB3OEEOZOY4LTVGFHTANCNFSM47PHCGKA
.

Nice thanks! Yeah if you have some sample or setup that reproduces the issue that you can share with me then I can also take it from there. But maybe those logs will already be enough :)

Hi Saelo!
Through hack --inspect=history, I successfully dumped the program which is fail to mutate to the .fuzzil.history file. I found that the main reason for the failure of mutating is in the CodeGenMutator's splice, after the splice, there will be repeated definitions of variables. I guess the root cause should be that the corpus generated by my Compiler does not ensure that the variable number of each instruction is in order, such as the following
// ===== [Program 67CCD11E-D966-4146-9113-D2D0A765CFD9] =====
// Corpus entry #341 on instance AA614686-1488-4F4E-9479-CCD1616C528E with Corpus type Corpus
v40 <- LoadBuiltin'print'
v24 <- LoadBoolean'true'
v38 <- LoadBuiltin'kWasmI32'
v22 <- LoadString'glob'
v20 <- LoadString'table'
v18 <- LoadString'wasm_mul'

The corpus generated by the ocaml compiler is in order, so the ocaml compiler has no similar problems.

So I think the bug here is that Code.isStaticallyValid does not correctly check for that, and probably this should also be documented more clearly in the docs. I'll take care of that!

Thanks a lot!
Samuel Groß @.***> 于2021年6月30日周三 下午9:49写道：

Ah great, yeah there are probably various bits of code, such as
Code.nextFreeVariable
https://github.com/googleprojectzero/fuzzilli/blob/f31ba8ee5f1d7d5f338a0d047a55d21b13494f7a/Sources/Fuzzilli/FuzzIL/Code.swift#L122,
that will misbehave if the variable numbers are not properly ordered.
So I think the bug here is that Code.isStaticallyValid does not correctly
check for that, and probably this should also be documented more clearly in
the docs. I'll take care of that!
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/googleprojectzero/fuzzilli/issues/218#issuecomment-871420965,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQB67X4AWIUMKBREEOBI4ALTVMOFDANCNFSM47PHCGKA
.

https://github.com/googleprojectzero/fuzzilli/commit/353d04a43baddd5df6410041cb09cf1a072ba741 Thanks again for reporting this! :)


Added #NODOORS

Added a new setting for disabling door generation.

A huge 'Thank you!' for your contribution.


Error while sending the message on the app

This was the error i encountered "Failed to sync all users. Status Code: 0 Exception: Code: generalException Message: An error occurred sending the request."

Hi @johnadetola ,
Thanks for raising the query. Could you please respond to the below queries.

Is this issue happening for all sending options?
Is this the first message?
What is the version of the app?
Please verify the configuration end-end  once again

Is this issue happening for all sending options?
I've tried most of the options, the same response.
Is this the first message?
I've not successfully send any message
What is the version of the app?
5.4
Please verify the configuration end-end  once again
I've done that.

Hi @johnadetola ,

Check whether the Supported type parameter for app registration is as per the deployment guide. Here is the reference.
Author-multiple
User-multiple
Graph-my organization

Could you please let us know the status of the issue otherwise we will make it to closure

I've checked that, the supported type is the same as suggested by you.

Hi @johnadetola,
Could you please share the screenshot of Company Communicator UI

Please respond to the above query otherwise we come to conclusion that the issue has been resolved and make it to closure.


Improve metadata integration efficiency - retain metadata state and configurable expiry

Repository metadata KPIs are stored in the client instance - in this case, if the repository supports expanded models. When metadata is stale, the next service operation that can make use of metadata will first attempt to fetch and refresh the client metadata state. The operation will then continue as normal.
Adds the optional metadataExpiry parameter to the ModelsRepositoryClientOptions ctor to configure when
the client should consider metadata repository KPIs stale.

API changes have been detected in Azure.IoT.ModelsRepository. You can review API changes here
API changes
-         public ModelsRepositoryClientOptions(ServiceVersion version = V2021_02_11);
+         public ModelsRepositoryClientOptions(ServiceVersion version = V2021_02_11, TimeSpan? metadataExpiry = null);
+         public static TimeSpan DefaultMetadataExpiry { get; }
+         public TimeSpan MetadataExpiry { get; }


Ensure events are readonly after they have been saved

This handles some cases, but unfortunately methods like #delete do not check this value.
I would be happy to add code that disables all of the dangerous methods as well, WDYT?

I just extracted a tiny gem for this that I could use in this PR instead https://rubygems.org/gems/activerecord-immutable

Nice looking gem. I think using that in this PR makes sense to me.


change class name to more specific one

Hey Justin! Nice to meet you! I'm Ramiro, an argentinian developer working on a project that uses react and ruby on rails. I'm using your library that's awesome, but I'm having a problem: other library uses a class called overlay and is in conflict with yours. Does exist the possibilty on change the name? Or something in mind?
I really appreciate your answer!
Thank you very much!
Ramiro

Thanks for the suggestion, added this in v1.6.1!


[graphql] Add DateTime tests

Description
Add tests to check if DateTime returns the right milliseconds granularity.
Test Plan
Added new test for DateTime.

If your changes are not user-facing and not a breaking change, you can skip the following section. Otherwise, please indicate what changed, and then add to the Release Notes section as highlighted during the release process.
Type of Change (Check all that apply)

[ ] protocol change
[ ] user-visible impact
[ ] breaking change for a client SDKs
[ ] breaking change for FNs (FN binary must upgrade)
[ ] breaking change for validators or node operators (must upgrade binaries)
[ ] breaking change for on-chain data layout
[ ] necessitate either a data wipe or data migration

Release notes

Thanks for adding these tests @Ştefan, your results seem conclusive to me! What I think is happening is that the milliseconds part is entirely omitted if the timestamp corresponds to a whole number of seconds, see, e.g. here:
https://github.com/MystenLabs/sui/blob/main/crates/sui-graphql-e2e-tests/tests/transactions/system.exp#L1270
What I thought I noticed was that when I advanced the clock by sub-second increments in the system.move E2E tests, it wasn't registering a different time for things like the checkpoint commit timestamp in ConsensusCommitPrologueTransaction, but now I'm thinking I just messed up the multiplier for bumping by a certain number of milliseconds instead of nanoseconds, (I thought I bumped it up in increments of 1000 from ns to ms to us to s, but chances that I can't count to 9 are ...non-zero).
I think the only further action here is:
* To clarify the docs on the DateTime scalar to mention that the milliseconds part is optional (may be omitted if all zero), and maybe

* To add a test to doubly-confirm that as well, in your excellent test suite.

Accepting with those comments in mind!

Looking at the system.move file, the 42xxx ns is actually 42s. In these tests I went from ns, to ms, to seconds, to hours, to make sure we get the right stuff.


productID field

It looks like the new radar (and possibly the old one too) provide a productID field in the creation body. In one of my tests with iOS & Accessibility as the product and area, this value was 210. The component field also still exists. Inspecting the radar page, I don't see where this 210 value came from. I'm going to punt on this for now.

productID comes from each dictionary from the response from https://bugreport.apple.com/developerUI/problem/getProductFullList

Any idea how that differs from the component?

As far as I know, nope.


hi

if i put amass -brute -df domain.file , it is just brute domain.file right? or brute and searching domain on internet

The flags you identified require the tool to be used as follows: amass -brute -df file_with_domains.txt


Simplify serialization by removing redundant PrimitiveScalarValue

~Draft as it builds on https://github.com/apache/arrow-datafusion/pull/3547~
Which issue does this PR close?
Re https://github.com/apache/arrow-datafusion/issues/3531
This scratches an itch I had while last working with this code
Rationale for this change
While working on https://github.com/apache/arrow-datafusion/issues/3531 I noticed that PrimitiveScalarType duplciated the functionality of DataType -- this makes working with the protobuf serialization code, which is already hard, even harder.
As the DataFusion community grows, we need to keep our code as easy to work with as possible to make reviews as well as new contirbutions easy.
What changes are included in this PR?

Remove PrimitiveScalarValue and use the existing Arrow DataType

Are there any user-facing changes?
No

This looks fine for me, not sure what stability guarantees we provide on this API. Theoretically we should go through a period of supporting both, but I don't know if this is worth the effort

Current answer I believe is "we provide no guarantees" (even though in theory we could provide a backwards compatible APIs). For example https://github.com/apache/arrow-datafusion/pull/3547 was likely not backward compatible but yet did not seem to cause changes.
cc  @andygrove  and @yahoNanJing  I think Ballista is the other major user of this API

I plan to merge this tomorrow unless someone has additional comments or would like more time to review


修复multipleSelect 查询过滤使用关联模型时页面刷新后无法显示选中内容

模型表格 multipleSelect  查询过滤中使用关联模型
$filter->in('levels.id', 'levels')
       ->multipleSelect(Level::all(['id', 'name'])->pluck('name', 'id'));

生成的 blade 模板中 $name 为 levels[id] ，$column 为 levels.id
请求中获取不到 $name 对应的字段，而无法正确显示选中内容

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

????

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

想关就关吧


Modify support

would be great to have more control within the event loop of updating kwargs sent to prompt_toolkit (eg allowing completer to be defined) and for access to answers
this PR allows this via modifyKwargs and modifyAnswers parameters which take function as value.  It could be possible to use filter to achieve same affect as modifyAnswers, but this might be less of a hack
documentation and tests included, however although it works - i cant get test to fully work with completer (involves TAB key) - but it does actually work :-)
hopefully this is of interest

Hi Roger, thank you for your PR. I do not really see how you need a new TAB control. I hope you understand that I can not accept a PR that breaks the tests.


Create data pipeline issues

Please create and populate issues for the following tasks:

[ ] - Identify access method for LCS and CHP datasets

Items we can provide development support on:

[ ] - Develop python or Snowflake scripts for processing XML files into structured data for all districts
[ ] - Develop python or Snowflake scripts for processing XML files into structure data for configuration files

Items we cannot provide development support on, guidance only:

[ ] - Create data relay pipeline for D7 VDS data
[ ] - Create data relay pipeline for the rest of the districts data
[ ] - Create data relay pipeline for VDS configuration XML files
[ ] - Create data relay pipeline for LCS data

@ian-r-rose if it would help speed things up for you I can create placeholders for each of these for you to fill in details on later. Just let me know.

Closing as complete


SetHideFlags now ORs HideFlags rather than setting. Changed defaults …

…work better with default Usd Asset import settings. The old code overrode the Hide settings for instance Masters, making them visible in the Hierarchy scene graph.

If it’s set to / then donotsave is applied to the root, meaning the root
will disappear on save/reload.
Probably should be changed to use mode regex and “/.+”
On Sat, Apr 6, 2019 at 08:19 Jeremy Cowles<EMAIL_ADDRESS>wrote:

@jcowles commented on this pull request.
In package/com.unity.formats.usd/Samples/ImportProcessor/SetHideFlags.cs
https://github.com/Unity-Technologies/usd-unity-sdk/pull/65#discussion_r272798898
:

@@ -24,21 +24,38 @@ public class SetHideFlags : RegexImportProcessor, IImportPostProcessHierarchy {

 public HideFlags hideFlagsSettings = HideFlags.DontSave;

public enum Operation {Overwrite, LogicalAND, LogicalOR, LogicalXOR}
[Tooltip("The logical operator to use when setting the flags. In most cases Or will produce expectd results")]

Typo in "expectd"
In package/com.unity.formats.usd/Samples/ImportProcessor/SetHideFlags.cs
https://github.com/Unity-Technologies/usd-unity-sdk/pull/65#discussion_r272798908
:

   }

 }

 void Reset() {

 matchExpression = "/";

 matchExpression = "/World";

Why "/World"?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Unity-Technologies/usd-unity-sdk/pull/65#pullrequestreview-223544804,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AshqAMgr0Z8JRvsYXcQxNFez-qHhmWQVks5veLr1gaJpZM4b3gtZ
.

Sorry for the delay, LGTM!


[BUG] After two nodes Harvester upgrade to v1.2.0-rc4, all virtual machines are in shutdown (off) state

Describe the bug

After we successfully upgrade from v1.1.2 to v1.2.0-rc4, all virtual machines are in Off state
(Except the RKE1 guest cluster VM)

Before upgrade, one VM in running and anther in off state, after upgrade all vms keep in off state
This would probably affect some running service.

To Reproduce
Steps to reproduce the behavior:

Create two VM, one in running and another in off state
Provision a RKE1 cluster from external Rancher
Create backup and snapshot for VMs
Upgrade to v1.2.0-rc4
Check the VM state after the upgrade complete

Expected behavior

All VMs should keep it's original status after the upgrade complete.
Support bundle

supportbundle_fb7d82fb-6347-4d68-ad9f-923d4996b5f1_2023-07-26T13-58-39Z.zip
Upgrade log
hvst-upgrade-bc7g6-upgradelog-archive-2023-07-26T13-26-18Z.zip
Environment

Harvester ISO version: v1.1.2 to v1.2.0-rc4
Underlying Infrastructure (e.g. Baremetal with Dell PowerEdge R630): Two nodes

The VM 'running` is pined to node1. It will be shut down during the upgrade (non-migratable):
      nodeSelector:
        kubernetes.io/hostname: node1

As an enhancement plan, we may show non-migratable VMs to users before an upgrade.

Thanks for the check. I created an enhancement ticket #4321 to suggest notifying user which types of virtual machines would be shutdown during upgrade.


Implement the wit-version keyword.

Implement the wit-version keyword, requiring the version string to be "0xa" for now. Once the language reaches a level of stability, this may be used to manage backwards-incompatible langauge changes.
This implements WebAssembly/component-model#126.
Opening as a draft for now, until it's clear how the component-model PR will be resolved.

Closing, as the relevant files have moved to wasm-tools. I'll port the changes over depending on the outcome of the component-model repo PR: https://github.com/WebAssembly/component-model/pull/126.


Rust rfc#88 was closed w/o implementation

https://github.com/pingcap/tikv/blob/716633c4c417da126b9d0ada9dd1ad6faec2d64f/src/util/macros.rs#L30-L31
see: https://github.com/rust-lang/rfcs/pull/88

Thank you for the notice @odeits ! I will update it soon.


Don't show an error message when receiving ContentModified error

ContentModified error response is send by the server if, during computation of the normal response, the server received DocumentChanged notification. It is useful for the servers who maintain only a single, "current" copy of all the state. ContentModified is not really an "exceptional" error, it is expected to happen on a regular basis.
Currently, VS Code shows a user-visible error popup for this cases, but the correct behavior would be to either silently use the default "empty" response, or to retry the request.
cc https://github.com/Microsoft/vscode-languageserver-node/commit/e72e16ed9e57ea569b5f1265614d53789f916997

Retrying the request doesn't make sense for positional requests since it is hard to tell if the position still makes sense. The right approach would be to have a content modified handler in the client so that this can be controlled by the extension author.


downloader: Add optional ability to authenticate when getting master playlist

Authenticating to a particular twitch account can give benefits, most notably not being served ads.

In the interests of time, merging without waiting for review. Post-merge review still welcome.


⚠ Minecraft has degraded performance

In 27c3aa5, Minecraft (https://www.minecraft.net) experienced degraded performance:

HTTP code: 200
Response time: 1199 ms

Resolved: Minecraft performance has improved in 2052d96.


Discuss performance tips around using Set<T> and how contexts can be optimised in heavier scenarios

As per this blog post for EF Core 6 Preview 5 there is no mention of Set<T>() in either the performance or querying section of the docs.
A small remark in the post:

Performance tip: you can avoid the overhead of DbSet initialization by using an alternate approach such as the context.Set() API call.

is the first time I have seen this be mentioned publicly in a wide-reaching post.
This should be expanded upon, or at the very least dumped, into a page on the performance docs, please!

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: c89e1e66-3c90-23d9-2c92-568e8c57d0ad
Version Independent ID: 26a9ec30-c290-f9bb-9d65-50d4e785872d
Content: Introduction to Performance - EF Core
Content Source: entity-framework/core/performance/index.md
Product: entity-framework
Technology: entity-framework-core
GitHub Login: @roji
Microsoft Alias: avickers

I would love to see something along this nature as well. There are obvious higher-level performance tips such as context pooling, no-tracking when needed, etc, but it would be fascinating to read some of the smaller ones that - while not needed for everyday usage - give us insights into how we can get the most out of EF.

I agree as well, it'd be nice to know whether or not Set<T>() should be used, especially with null reference types enabled.

Seconding, as I've been using .Set<T>() over DbSet<T> properties for quite a while now. Some folks tell me I'm nuts, and if so, I'd like to know it.
I can't imagine how it's not a performance boost, though, even if it's rather small. Sure, you're not gonna see much benefit from it, compared to other optimizations, but it's also basically costless. The only pitfall I'm aware of is that EF can miss entity types, if you don't explicitly add them all in OnModelBuilding().

If you have settable DbSet properties, the DbContext constructor will use reflection to find them and initialize them with DbSets. For very perf-sensitive application, this could slow things down when creating new DbContext instances.
However, there are a lot of other things going on when creating new DbContext instances, so perf-sensitive applications should strongly consider using DbContext pooling. Doing so means the DbContext setup cost (including the setting of DbSet properties) is only paid once per context instance, and from that point the instance is reused.
From a perf perspective, this is better than calling Set<T>() every time you execute a query, since doing that adds an additional dictionary lookup to find the set each time. When a DbSet property is used, it has already been set and no lookup is necessary.
So for ideal perf, use DbContext pooling and have DbSet properties. However, it's important that understand that these difference are very likely to be negligible for typical, real-world application where networking and database processing times are likely to dominate.

@roji Would you consider a PR from myself (or anyone who is faster than me) that puts this into the docs, obviously edited/reworded to fit with the style of the docs?

I generally believe the docs shouldn't be overloaded with details that don't matter to the average EF user (and I think this is a good example). However, a point in the reducing runtime overhead section could make sense.

@patrickklaeren I'd love for you to try different patterns for your particular application and post back the performance differences.

I agree with @ajcvickers - always good to see concrete numbers with perf. If you show it's significant for a real-world application, it may make sense to highlight this more prominently.


feat(run summary): Add whether turbo detected monorepo or not

While debugging another issue, I came across a confusing behavior where it seemed like turbo was operating in single package mode. I added this bit of information in to get some visibility in, and figured it was a good feature for everyone.

Not a huge deal, but maybe we should have this as a mode field so it's more extensible?

@NicholasLYang that feels to general, but let me know if you have a proposal


Update default moderator permissions

Moderators should be able to update topics and avatars, but should not be able to make rooms encrypted.
I think not a blocker for FOSDEM 2022 as we can run a script.

Duplicate of #98


Prometheus exporter's Collect method copies ViewData

Fixes #402
The previous code mistakenly used an unprotected copy
of the collector's viewData(a map) which is concurrently
written to, when a program adds views.
This commit makes a copy of what is present in that
viewData up until this point in time, that Collect is invoked.

Thank you for the review @songy23!


The uninitialized voter which comes from splitting may lead to two same-term raft leaders

Bug Report
Like #3572 and #8381, there is another case that there may be two same-term leaders.
See this case:

region A has peer 1, 2, 3 on store 1, 2, 3, respectively
peer 1 is leader and peer 3 is isolated
1 proposes split cmd and region B is created.
1 proposes two conf change to remove peer 3 and add peer 4
peer 3's network recovers and it will get a gc response from 1 or 2 then it will destroy itself.

So the peer of region B on store 3 is not created because peer 3 of region A does not apply the split cmd.
But the other peer of region B will consider this peer is a voter.
As a result, the problem of two same-term leaders may occur.

/cc @BusyJay @NingLin-P

/cc @BusyJay @NingLin-P

To fix it permanently, maybe we need to store hard state. It may change some assumptions but the change should be small.

To fix it permanently, maybe we need to store hard state. It may change some assumptions but the change should be small.


Cannot execute repl, Resolution failed for ammonite 6:2.3.8-65-0f0d597f

$ mill -i  anorm-async[2.13.6].repl
Preparing Java 11.0.12 runtime; this may take a minute or two ...
Compiling /home/simao/code/anorm-async/build.sc
[27/35] anorm-async[2.13.6].compile
Compiling compiler interface...
[info] compiling 1 Scala source to /home/simao/code/anorm-async/out/anorm-async/2.13.6/compile/dest/classes ...
[warn] 2 deprecations (since 2.13.0); re-run with -deprecation for details
[warn] one warning found
[info] done compiling
[31/35] anorm-async[2.13.6].resolvedAmmoniteReplIvyDeps | Downloading [2/2] artifacts (~0/0 bytes)
1 targets failed
anorm-async[2.13.6].resolvedAmmoniteReplIvyDeps
Resolution failed for 1 modules:
--------------------------------------------
  com.lihaoyi:ammonite_2.13.6:2.3.8-65-0f0d597f
	not found: /home/simao/.ivy2/local/com.lihaoyi/ammonite_2.13.6/2.3.8-65-0f0d597f/ivys/ivy.xml
	not found: https://repo1.maven.org/maven2/com/lihaoyi/ammonite_2.13.6/2.3.8-65-0f0d597f/ammonite_2.13.6-2.3.8-65-0f0d597f.pom

I am running mill 0.9.9 using the bootstrap script, I got this error before and after deleting ./out and ~/.mill.

This is most probably not a bug. The following answer might help: https://github.com/com-lihaoyi/mill/discussions/1396#discussioncomment-969512

Thanks @lefou , I had the same issue and the solution you proposed worked for me. Also, I discussed the problem in gitter https://gitter.im/lihaoyi/mill
That being said I disagree that this is not a bug. This is very frustrating for the user and really hurt the credibility of the project as a serious / mature competitor to sbt.
In my opinion, this is at least a usability / ux issue that need to be fixed. As a user, I just want mill to work.
Proposed solution design
Maybe try to add a default value for ammonite if the default doesn't work. Or an incremental search for an ammonite version that works.

Having a default looks convenient at first sight, but in general, default versions tend to not age well.
In this particular case it's rather hard to come up with a good default, as the Ammonite version highly depends on the already user-configured Scala version! What ever logic to retrieve a valid (if any) Ammonite version you have in mind, it will be definitely not easy to guess for other users and most likely not reproducible over time, which leads to a frustrating users experience too.
Therefore I think, it's much better to simply require an explicitly set Ammonite version, as we do for other versions like the scalaVersion.
Implementation wise, we should deprecate the currently defined default and print some warning in case a user starts the repl without a overridden ammoniteVersion. Also some documentation is required.

@lefou could one solution be to closely follow the latest Scala/Ammonite releases with a corresponding Mill release?
Ammonite already releases a new version to match every Scala minor version, and if Mill pulls that in, at least we'll be able to guarantee that if you use the latest Mill release with the latest Scala version, Ammonite will work out of the box. Scala doesn't release all that frequently, so matching releases wouldn't be too much of a maintenance overhead
Now if someone uses an old version of Mill with new version of Scala, there's not much we can do there, but at least we can make the "latest latest" case work.

@lihaoyi I understand the reasoning, but technically, it's not even needed to cut a new mill release for each new Scala or Ammonite release. And if it is not needed, why create new releases which will put more stress to Mill devs and users. It's the exact reason why this is configurable.
Each release means a lot of work, the smallest part is on the mill devs (e.g. releasing, testing, documenting, etc). The full storied only starts there and continues with work for plugin developers and users and so on.

Also, closely following dependent versions is to-date very easy for open source projects by using @scala-steward . E.g. for Ammonite, just define an ivy dependency an Ammonite and refer to the ammoniteVersion and voila, you'll receive a PR as soon a new Ammonite release is out.

Yeah it's not otherwise necessary, it would be some additional operational overhead and churn just to get this working seamlessly.
I'll leave it to your judgement however you would like to decide on the tradeoff.

In the past it was not even possible to keep up with the latest versions of Scala and Ammonite. Also we currently have a stable branch for 0.9.x but development happens on main branch for upcoming 0.10, which has no stable API yet and hence no available external plugins. As always, the devil is in the details.
I'd favor to just try to provide a better error message to guide the user in case the built-time Ammonite version isn't available for the selected Scala runtime.


How to calculate accuracy like Top 1,5 and 10 results with visualization ?

You should do it in a manual way. DeepFace just returns candidate face information.


Fails to deploy dd-agent on bastion

The bastion ansible isn't executed with a secrets.yml file, this is only in the cideploy user. This means it fails on the bastion with

fatal: [contrasjc-bastion]: FAILED! => {"changed": false, "failed": true, "msg": "AnsibleUndefinedVariable: 'secrets' is undefined"}

We probably need to store secrets outside the cideploy user, maybe ssh keys as well (although ssh keys are less important now we're on a public github).

@j2sol ping

Pretty sure this got fixed. Re-open if not.


Add physicalInputDataSize to BasicQueryStats.

As mention in https://github.com/prestosql/presto/pull/4076, split the change into two part, this is the first commit add physicalInputDataSize to basic query stats

@Praveen2112 Can you take a look at this pull request.

Merged !! Thanks for working on this


Do not log on explicit no unsafe again

Motivation:
Users should not see a scary log message when Netty is initialized if
Netty configuration explicitly disables unsafe. The log message that
produces this warning was previously guarded but the guard was
lost.
Modifications:
This commit brings back the guard against the scary log message if
unsafe is explicitly disabled.
Result:
No log message is produced when unsafe is unavailable because Netty was
told to not look for it.
Relates #5624, relates #6568

@jasontedor LGTM... Thanks a lot again. Will merge once the CI finishes

Thanks @normanmaurer!

Cherry-picked into 4.1 (9a0fd3a7b892438fccff3d63946756160a8647fe) and 4.0 (bf2ab608ec)

Thanks for merging @normanmaurer.


Segfault on using -psm 0 when using fast eng.traineddata

Environment

Tesseract Version: 4.00 alpha
Commit Number: 1b0379c
Platform: Presumably all - found on Ubuntu 17.04, confirmed on macOS Sierra

Current Behavior:
When using eng.traineddata from tessdata_fast in -psm 0 mode Tesseract crashes for all input files. Example:
b095cb28b6c868b99d19e1c64b48a626bc4cb944  osd.traineddata
31abd495e0f719db4f524c447e9d855124a0b0d6  eng.traineddata

$ tesseract -psm 0 testing/phototest.tif stdout
Segmentation fault

Behaviour is the same using tessdata_best.
After replacing with tessdata/eng.traineddata, OSD works fine:
b095cb28b6c868b99d19e1c64b48a626bc4cb944  osd.traineddata
cdcfae0c5c272b5b2f0406cc91ac5d022f7df7f4 eng.traineddata

$ tesseract -psm 0 testing/phototest.tif stdout
Page 1
Page number: 0
Orientation in degrees: 0
Rotate: 0
Orientation confidence: 15.98
Script: Latin
Script confidence: 460.00

I discovered this in the Ubuntu 17.04 PPA (ppa:alex-p/tesseract-ocr) and replicated it on macOS tesseract built from source.
Stack trace
Stack trace (Linux version)

#0  0x00007f5e14a5494d in tesseract::Classify::CharNormClassifier(TBLOB*, tesseract::TrainingSample const&, ADAPT_RESULTS*) () from /usr/lib/libtesseract.so.4
#1  0x00007f5e14a55885 in tesseract::Classify::DoAdaptiveMatch(TBLOB*, ADAPT_RESULTS*) () from /usr/lib/libtesseract.so.4
#2  0x00007f5e14a55f89 in tesseract::Classify::AdaptiveClassifier(TBLOB*, BLOB_CHOICE_LIST*) () from /usr/lib/libtesseract.so.4
#3  0x00007f5e14973c82 in os_detect_blob(BLOBNBOX*, OrientationDetector*, ScriptDetector*, OSResults*, tesseract::Tesseract*) ()
   from /usr/lib/libtesseract.so.4
#4  0x00007f5e1497413b in os_detect_blobs(GenericVector<int> const*, BLOBNBOX_CLIST*, OSResults*, tesseract::Tesseract*) () from /usr/lib/libtesseract.so.4
#5  0x00007f5e1497453d in os_detect(TO_BLOCK_LIST*, OSResults*, tesseract::Tesseract*) () from /usr/lib/libtesseract.so.4
#6  0x00007f5e14974782 in orientation_and_script_detection(STRING&, OSResults*, tesseract::Tesseract*) () from /usr/lib/libtesseract.so.4
#7  0x00007f5e14943f8f in tesseract::TessBaseAPI::DetectOS(OSResults*) ()
   from /usr/lib/libtesseract.so.4
#8  0x00007f5e149440b9 in tesseract::TessBaseAPI::DetectOrientationScript(int*, float*, char const**, float*) () from /usr/lib/libtesseract.so.4
#9  0x00007f5e149441b1 in tesseract::TessBaseAPI::GetOsdText(int) ()
   from /usr/lib/libtesseract.so.4
#10 0x00007f5e1494bcd4 in tesseract::TessOsdRenderer::AddImageHandler(tesseract::TessBaseAPI*) () from /usr/lib/libtesseract.so.4

The OSD module is based on the legacy engine. 'fast' and 'best' traineddatas was trained for LSTM only.

tesseract -psm 0 testing/phototest.tif stdout

The eng is the default lang even when you use --psm 0.

Try this:

tesseract --psm 0 -l osd testing/phototest.tif stdout

The explicit -l osd works around the issue.
Although it shouldn't crash even if the arguments are invalid.

@stweil Should the program use -l  osd by default internally for 4.0.0 when --psm 0 is used?

Should the program use -l  osd by default internally for 4.0.0 when --psm 0 is used?

Yes, IMO.

I think this was fixed in commit 27ce472666a6332f38888cd6855e1162fb82c729, so the issue can be closed.


[REGRESSION]: Unable to screenshot scrollbar on Firefox headless true

Version
1.45.3
Steps to reproduce
We ran into a problem with my team with regression tests in Firefox. Firefox hides scrollbars in headless mode. If I set headless to false in the configuration, the scrollbar is visible but the images are not stable. Is there a way to override the css to show the scrollbars in headless mode or even better an option like ignoreDefaultArgs: ['--hide-scrollbars'] for chrome?
Expected behavior
Having screenshots with scrollbars in firefox as well
Actual behavior
It's only possible to have scrollbars in headless set to false but that breakes our screenshots

Hi @kacperczerwinski , is this issue specific azure playwright service or this is generic playwright issue. this github repo is for issue related to azure playwright service. You can log generic playwright issue on https://github.com/microsoft/playwright

Hi @puagarwa, perfect. Thanks for your fast response

Hi @puagarwa, perfect. Thanks for your fast response


Jspm typescript

Add karma hack for karma-jspm issue
switch to bower-angular-mocks to fix loading issue with missing
angular object
update typings

You need to create pull request to jspm-typescript branch, not to master

manually merged to jspm-typescript branch. You can now delete your fork, thx !


Use ip@5 if found instead of sp as optional NCEP spectral library

This resolves https://github.com/GEOS-ESM/GSIbec/issues/65. I've tested compiling with either ip@5 loaded, or<EMAIL_ADDRESS>loaded, or none of them, and it compiled successfully each time. I didn't run any tests with it, though.

@AlexanderRichert-NOAA FYI

Thank you @rtodling !


Help with API to update

Do you have a example how be updated some data using the API? Please your help

You can look at the web interface or Traccar Manager apps for API examples.

Thanks but there is not a clear example of how to use UPDATE with SWAGGER API.
Somone in this forum has an example in PHP?
Im using Guzzle to interact with Laravel. By example this is how i add a Geofence:
$client = new Client([
'base_uri' => 'http://www.mytraccarserver.cl:8082/api/',
'timeout'  => 2.0,
]);
$response = $client->request('POST', 'geofences', ['json' => ['name' => $input["name"], 'area' => $input["area"]],'auth' => ['username', 'pass']]);

Update is the same as create except you have to use PUT verb instead of POST.


vcf convert plink format

The  vcf file produced by MapCaller  can't be converted into Plink formate file (map and ped), command: vcftools --vcf result.vcf  --plink --out plink,  Error: Require Genotypes in VCF file in order to output as PLINK. However, the problem don't happen in GATK software.   The example  vcf  in  MapCaller tutorial (ref.vcf and out.vcf) can't work too.
Eagerly looking forward to response, thanks very much again！

Hi, thanks for the information. MapCaller does output genotype for snv and indels. I think it is an issue of formatting the attribute fields. I'll work on it and figure out how to modify the output. Thank you again.

Hi, hsinnan75, thank you very much. Looking forward to good news!

Hi, I've fixed the problem. Please download the latest version (v<IP_ADDRESS>). Please try again and let me know if there are any other problems regarding the use of MapCaller. Thank you!!

I am glad to receive your messenge，and I'm using it as soon as possible. Thank you  again!


Fix predefined metrics not match in monitoring

Issue:
https://github.com/rancher/rancher/issues/18624
Related PR:
https://github.com/rancher/system-charts/pull/26

@thxCode since resource container doesn't have summary and details, could you update the metric, no need group by container name, and the legendFormat don't need [[container_name]] any more.

@aiwantaozi, Updated

LGTM

LGTM

LGTM

Rebased & Squashed

LGTM, restarting build bc it had an unrealted failure


Module of constants/enum for each id type

There are hundreds of courses, all with unique ids. Perhaps there is an API to query this and automatically generate them? For now a direct mapping exists for as many classes as possible.

Add mappings for: Class, Semester, Career, Class->Career

https://engineering.buffalo.edu/computer-science-engineering/undergraduate/courses/course-catalog.html
Can be used to extract few details about class options per course.

Partially done:

Python script created to pull a list of all pathways
Build script which automatically generates enums based on csv of course mappings

All SEAS Courses Schedule
https://cse.buffalo.edu/calendar/courses/


write errors while running the TechEmpower/Updates benchmark

When the Updates benchmark is run with Kitura-NIO, we see some intermittent write errors:
[2018-11-20T19:39:41.296Z] [ERROR] [HTTPServerResponse.swift:90 write(from:)] No channel available to write
[2018-11-20T19:39:41.297Z] [ERROR] [HTTPServerResponse.swift:123 end()] No channel available to end the response
[2018-11-20T19:39:41.297Z] [ERROR] [HTTPServerResponse.swift:90 write(from:)] No channel available to write
[2018-11-20T19:39:41.297Z] [ERROR] [HTTPServerResponse.swift:123 end()] No channel available to end the response

The errors come from here and here.

I see similar messages with Kitura-Net
[2018-11-21T06:40:39.364Z] [WARNING] [IncomingSocketHandler.swift:362 write(from:length:)] IncomingSocketHandler write() called after socket 18 closed
[2018-11-21T06:41:44.068Z] [WARNING] [IncomingSocketHandler.swift:362 write(from:length:)] IncomingSocketHandler write() called after socket 117 closed

This is a duplicate of https://github.com/IBM-Swift/Kitura-Benchmarks/issues/16 which has been closed now.


Windows Server 2019 Container (At least 3Gb memory needs to be available to)

Hi I'm trying to run a container with gmSA.
this error shows up ..
`
At least 3Gb memory needs to be available to the Container.
At C:\Run\start.ps1:21 char:5

throw "At least 3Gb memory needs to be available to the Container ...

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

CategoryInfo          : OperationStopped: (At least 3Gb me... the Contai
ner.:String) [], RuntimeException
FullyQualifiedErrorId : At least 3Gb memory needs to be available to the
Container.
Command:
New-NavContainer -accept_eula -accept_outdated -containerName demo -auth Windows -licenseFile "S:\share\my\lic.flf" -shortcuts Desktop -additionalParameters @('--security-opt credentialspec=file://containerhost.json') -imageName mcr.microsoft.com/businesscentral/onprem:cu1-at-ltsc2019
`

Same with -memoryLimit 4G
docker run --name demo --hostname demo -e Accept_eula=Y -e Accept_outdated=Y -v s:\share\my\:c:\run\my -e licensefile=c:\run\my\lic.flf -e Auth=Windows --security-opt "credentialspec=file://containerhost.json" mcr.microsoft.com/businesscentral/onprem:cu1-at-ltsc2019
If I do not use the --security-opt Option this error does not occur.
Any Idea on this?`

Can you show the command including the memory limit param and the full output? And also the result of docker inspect < containername >

Docker Run

NavContainerHelper

docker run ...

$hostname = "demo"
$imagename = "mcr.microsoft.com/businesscentral/onprem:cu1-at-ltsc2019"

docker run `
    -m 3G `
    --name $hostname `
    --hostname $hostname `
    -v S:\share\my\:c:\run\my `
    -e licensefile=c:\run\my\lic.flf `
    -e Accept_eula=Y `
    -e ACCEPT_OUTDATED=Y `
    -e Auth=Windows `
    -e locale=de-AT `
    --security-opt "credentialspec=file://containerhost.json" `
    -e enableSymbolLoading=Y `
    ${imagename}

Error:
At least 3Gb memory needs to be available to the Container.
At C:\Run\start.ps1:21 char:5
+     throw "At least 3Gb memory needs to be available to the Container ...
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (At least 3Gb me... the Contai
   ner.:String) [], RuntimeException
    + FullyQualifiedErrorId : At least 3Gb memory needs to be available to the
    Container.

Inspect:
[
    {
        "Id": "a0acde2553a523500469a0ad9a6f42ad5d3a8663270590cab6c00630dc53cf5d",
        "Created": "2018-12-03T07:53:19.7363336Z",
        "Path": "powershell",
        "Args": [
            "-Command",
            "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';",
            ".\\Run\\start.ps1"
        ],
        "State": {
            "Status": "exited",
            "Running": false,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 0,
            "ExitCode": 1,
            "Error": "",
            "StartedAt": "2018-12-03T07:53:27.044831Z",
            "FinishedAt": "2018-12-03T07:53:55.596416Z",
            "Health": {
                "Status": "unhealthy",
                "FailingStreak": 1,
                "Log": [
                    {
                        "Start": "2018-12-03T07:53:57.1268361Z",
                        "End": "2018-12-03T07:53:57.5579547Z",
                        "ExitCode": -1,
                        "Output": "container a0acde2553a523500469a0ad9a6f42ad5d3a8663270590cab6c00630dc53cf5d encountered an error during CreateProcess: failure in a Windows system call: The requested virtual machine or container operation is not valid in the current state. (0xc0370105) extra info: {\"CommandLine\":\"powershell .\\\\Run\\\\HealthCheck.ps1\",\"WorkingDirectory\":\"/\",\"Environment\":{\"ACCEPT_OUTDATED\":\"Y\",\"Accept_eula\":\"Y\",\"Auth\":\"Windows\",\"COUNTRYURL\":\"https://nav2016wswe0.blob.core.windows.net/dvd/2801b53a-9f04-4bcc-86d7-49bd43fc54fa\",\"CountryCode\":\"at\",\"DatabaseInstance\":\"SQLEXPRESS\",\"DatabaseName\":\"Cronusat\",\"DatabaseServer\":\"localhost\",\"IsBcSandbox\":\"N\",\"NAVDVDURL\":\"https://nav2016wswe0.blob.core.windows.net/dvd/06373ccb-644a-47dd-830d-5f2ac9f81fb9\",\"VSIXURL\":\"https://nav2016wswe0.blob.core.windows.net/dvd/al-2.0.56865.vsix\",\"enableSymbolLoading\":\"Y\",\"licensefile\":\"c:\\\\run\\\\my\\\\lic.flf\",\"locale\":\"de-AT\"},\"CreateStdInPipe\":true,\"CreateStdOutPipe\":true,\"CreateStdErrPipe\":true,\"ConsoleSize\":[0,0]}"
                    }
                ]
            }
        },
        "Image": "sha256:032552538928bc1031480d3f226c1a0f9da4c5cf7856c0ff2951c09ec0a8b1a0",
        "ResolvConfPath": "",
        "HostnamePath": "",
        "HostsPath": "",
        "LogPath": "S:\\docker\\containers\\a0acde2553a523500469a0ad9a6f42ad5d3a8663270590cab6c00630dc53cf5d\\a0acde2553a523500469a0ad9a6f42ad5d3a8663270590cab6c00630dc53cf5d-json.log",
        "Name": "/demo",
        "RestartCount": 0,
        "Driver": "windowsfilter",
        "Platform": "windows",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [
                "S:\\share\\my\\:c:\\run\\my"
            ],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "default",
            "PortBindings": {},
            "RestartPolicy": {
                "Name": "no",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": [
                "credentialspec=file://containerhost.json"
            ],
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 0,
            "ConsoleSize": [
                18,
                162
            ],
            "Isolation": "process",
            "CpuShares": 0,
            "Memory":<PHONE_NUMBER>,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": [],
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DeviceCgroupRules": null,
            "DiskQuota": 0,
            "KernelMemory": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": false,
            "PidsLimit": 0,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0
        },
        "GraphDriver": {
            "Data": {
                "dir": "S:\\docker\\windowsfilter\\a0acde2553a523500469a0ad9a6f42ad5d3a8663270590cab6c00630dc53cf5d"
            },
            "Name": "windowsfilter"
        },
        "Mounts": [
            {
                "Type": "bind",
                "Source": "s:\\share\\my",
                "Destination": "c:\\run\\my",
                "Mode": "",
                "RW": true,
                "Propagation": ""
            }
        ],
        "Config": {
            "Hostname": "demo",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": true,
            "AttachStderr": true,
            "ExposedPorts": {
                "1433/tcp": {},
                "443/tcp": {},
                "7045/tcp": {},
                "7046/tcp": {},
                "7047/tcp": {},
                "7048/tcp": {},
                "7049/tcp": {},
                "80/tcp": {},
                "8080/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "licensefile=c:\\run\\my\\lic.flf",
                "Accept_eula=Y",
                "ACCEPT_OUTDATED=Y",
                "Auth=Windows",
                "locale=de-AT",
                "enableSymbolLoading=Y",
                "DatabaseServer=localhost",
                "DatabaseInstance=SQLEXPRESS",
                "DatabaseName=Cronusat",
                "NAVDVDURL=https://nav2016wswe0.blob.core.windows.net/dvd/06373ccb-644a-47dd-830d-5f2ac9f81fb9",
                "VSIXURL=https://nav2016wswe0.blob.core.windows.net/dvd/al-2.0.56865.vsix",
                "IsBcSandbox=N",
                "CountryCode=at",
                "COUNTRYURL=https://nav2016wswe0.blob.core.windows.net/dvd/2801b53a-9f04-4bcc-86d7-49bd43fc54fa"
            ],
            "Cmd": [
                "powershell",
                "-Command",
                "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';",
                ".\\Run\\start.ps1"
            ],
            "Healthcheck": {
                "Test": [
                    "CMD",
                    "powershell",
                    ".\\Run\\HealthCheck.ps1"
                ],
                "Interval":<PHONE_NUMBER>0,
                "Timeout":<PHONE_NUMBER>0
            },
            "ArgsEscaped": true,
            "Image": "mcr.microsoft.com/businesscentral/onprem:cu1-at-ltsc2019",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {
                "country": "at",
                "created": "201811291319",
                "cu": "cu1",
                "eula": "https://go.microsoft.com/fwlink/?linkid=861843",
                "legal": "http://go.microsoft.com/fwlink/?LinkId=837447",
                "maintainer": "Dynamics SMB",
                "nav": "",
                "osversion": "10.0.17763.134",
                "tag": "<IP_ADDRESS>",
                "version": "13.1.25940.0"
            }
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "a0acde2553a523500469a0ad9a6f42ad5d3a8663270590cab6c00630dc53cf5d",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {},
            "SandboxKey": "a0acde2553a523500469a0ad9a6f42ad5d3a8663270590cab6c00630dc53cf5d",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "nat": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "d40a6ebc4cbe5447d6abfcb51ea96bba69a76aeab56dc5c5c2cb2f6ff1c0db72",
                    "EndpointID": "",
                    "Gateway": "",
                    "IPAddress": "",
                    "IPPrefixLen": 0,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "",
                    "DriverOpts": null
                }
            }
        }
    }
]

navcontainerhelper

Import-Module navcontainerhelper

$hostname = "demo"
$imagename = "mcr.microsoft.com/businesscentral/onprem:cu1-at-ltsc2019"

New-NavContainer -containerName $hostname `
-memoryLimit 3G `
-additionalParameters @('--security-opt credentialspec=file://containerhost.json') `
-licenseFile S:\share\my\lic.flf `
-accept_eula `
-accept_outdated `
-doNotExportObjectsToText `
-auth Windows `
-clickonce `
-imageName ${imagename}

Error:
At least 3Gb memory needs to be available to the Container.
At C:\Run\start.ps1:21 char:5
+     throw "At least 3Gb memory needs to be available to the Container ...
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (At least 3Gb me... the Contai
   ner.:String) [], RuntimeException
    + FullyQualifiedErrorId : At least 3Gb memory needs to be available to the
    Container.

Inspect:
[
    {
        "Id": "eb018ac8305d784824c285da82a69e6f7cdf3bfc76370944c34261035c397732",
        "Created": "2018-12-03T08:03:11.0981348Z",
        "Path": "powershell",
        "Args": [
            "-Command",
            "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';",
            ".\\Run\\start.ps1"
        ],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 4484,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2018-12-03T08:04:52.2322691Z",
            "FinishedAt": "2018-12-03T08:04:44.9132511Z",
            "Health": {
                "Status": "starting",
                "FailingStreak": 0,
                "Log": []
            }
        },
        "Image": "sha256:032552538928bc1031480d3f226c1a0f9da4c5cf7856c0ff2951c09ec0a8b1a0",
        "ResolvConfPath": "",
        "HostnamePath": "",
        "HostsPath": "",
        "LogPath": "S:\\docker\\containers\\eb018ac8305d784824c285da82a69e6f7cdf3bfc76370944c34261035c397732\\eb018ac8305d784824c285da82a69e6f7cdf3bfc76370944c34261035c397732-json.log",
        "Name": "/demo",
        "RestartCount": 4,
        "Driver": "windowsfilter",
        "Platform": "windows",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [
                "C:\\ProgramData\\NavContainerHelper:C:\\ProgramData\\NavContainerHelper",
                "C:\\ProgramData\\NavContainerHelper\\Extensions\\demo\\my:C:\\Run\\my"
            ],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "default",
            "PortBindings": {},
            "RestartPolicy": {
                "Name": "unless-stopped",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": [
                "credentialspec=file://containerhost.json"
            ],
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 0,
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "process",
            "CpuShares": 0,
            "Memory":<PHONE_NUMBER>,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": [],
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DeviceCgroupRules": null,
            "DiskQuota": 0,
            "KernelMemory": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": false,
            "PidsLimit": 0,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0
        },
        "GraphDriver": {
            "Data": {
                "dir": "S:\\docker\\windowsfilter\\eb018ac8305d784824c285da82a69e6f7cdf3bfc76370944c34261035c397732"
            },
            "Name": "windowsfilter"
        },
        "Mounts": [
            {
                "Type": "bind",
                "Source": "c:\\programdata\\navcontainerhelper",
                "Destination": "c:\\programdata\\navcontainerhelper",
                "Mode": "",
                "RW": true,
                "Propagation": ""
            },
            {
                "Type": "bind",
                "Source": "c:\\programdata\\navcontainerhelper\\extensions\\demo\\my",
                "Destination": "c:\\run\\my",
                "Mode": "",
                "RW": true,
                "Propagation": ""
            }
        ],
        "Config": {
            "Hostname": "demo",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "1433/tcp": {},
                "443/tcp": {},
                "7045/tcp": {},
                "7046/tcp": {},
                "7047/tcp": {},
                "7048/tcp": {},
                "7049/tcp": {},
                "80/tcp": {},
                "8080/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "databaseServer=",
                "databaseInstance=",
                "securePassword=76492d1116743f0423413b16050a5345MgB8AGgAQgB5AEcAZwBlAGMAbwBQADcAVwBnAHIANABCAEcAcQA0AEYAUABRAEEAPQA9AHwAYQBkADAAOAAyADEANwA5ADUAMwAwADkAMgBhAGQAOQA2AGEAOAAxADIAYwAyADQAMQAyADcANwAxADgAMQAyAGMAMwAzADQAMQAxADEANgA5ADUAZABmADUANwBjADEANAAzAGQANwA4AGUAMQA5ADMAYQA5AGQAYgAyAGEAMAA=",
                "clickonce=Y",
                "auth=Windows",
                "ExitOnError=N",
                "locale=de-AT",
                "enableApiServices=Y",
                "accept_outdated=Y",
                "username=admin",
                "useSSL=N",
                "passwordKeyFile=c:\\run\\my\\aes.key",
                "accept_eula=Y",
                "licenseFile=c:\\run\\my\\license.flf",
                "removePasswordKeyFile=Y",
                "DatabaseName=Cronusat",
                "NAVDVDURL=https://nav2016wswe0.blob.core.windows.net/dvd/06373ccb-644a-47dd-830d-5f2ac9f81fb9",
                "VSIXURL=https://nav2016wswe0.blob.core.windows.net/dvd/al-2.0.56865.vsix",
                "IsBcSandbox=N",
                "CountryCode=at",
                "COUNTRYURL=https://nav2016wswe0.blob.core.windows.net/dvd/2801b53a-9f04-4bcc-86d7-49bd43fc54fa"
            ],
            "Cmd": [
                "powershell",
                "-Command",
                "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';",
                ".\\Run\\start.ps1"
            ],
            "Healthcheck": {
                "Test": [
                    "CMD",
                    "powershell",
                    ".\\Run\\HealthCheck.ps1"
                ],
                "Interval":<PHONE_NUMBER>0,
                "Timeout":<PHONE_NUMBER>0
            },
            "ArgsEscaped": true,
            "Image": "mcr.microsoft.com/businesscentral/onprem:cu1-at-ltsc2019",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {
                "country": "at",
                "created": "201811291319",
                "cu": "cu1",
                "eula": "https://go.microsoft.com/fwlink/?linkid=861843",
                "legal": "http://go.microsoft.com/fwlink/?LinkId=837447",
                "maintainer": "Dynamics SMB",
                "nav": "",
                "osversion": "10.0.17763.134",
                "tag": "<IP_ADDRESS>",
                "version": "13.1.25940.0"
            }
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "eb018ac8305d784824c285da82a69e6f7cdf3bfc76370944c34261035c397732",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {
                "1433/tcp": null,
                "443/tcp": null,
                "7045/tcp": null,
                "7046/tcp": null,
                "7047/tcp": null,
                "7048/tcp": null,
                "7049/tcp": null,
                "80/tcp": null,
                "8080/tcp": null
            },
            "SandboxKey": "eb018ac8305d784824c285da82a69e6f7cdf3bfc76370944c34261035c397732",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "nat": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "d40a6ebc4cbe5447d6abfcb51ea96bba69a76aeab56dc5c5c2cb2f6ff1c0db72",
                    "EndpointID": "711f8d0c2de794f65ba69c31d0f22d4a1e031905cb6d4be60e6a2bd4baae120d",
                    "Gateway": "<IP_ADDRESS>",
                    "IPAddress": "<IP_ADDRESS>",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "00:15:5d:10:3d:40",
                    "DriverOpts": null
                }
            }
        }
    }
]

OS Info when using navcontainerhelper
Windows PowerShell credential request.
Using Windows Authentication. Please enter your Windows credentials.
Password for user admin: **************
Host is Windows Server 10.0.17763.0 - ltsc2019
Using image mcr.microsoft.com/businesscentral/onprem:cu1-at-ltsc2019
Removing container demo
Removing C:\ProgramData\NavContainerHelper\Extensions\demo
Creating Nav container demo
Using license file S:\share\my\lic.flf
Version: 13.1.25940.0-at
Generic Tag: <IP_ADDRESS>
Container OS Version: 10.0.17763.134 (ltsc2019)
Host OS Version: 10.0.17763.0 (ltsc2019)
Creating container demo from image mcr.microsoft.com/businesscentral/onprem:cu1-at-ltsc2019
eb018ac8305d784824c285da82a69e6f7cdf3bfc76370944c34261035c397732
Waiting for container demo to be ready
At least 3Gb memory needs to be available to the Container.

Additional Information:
This machine is the default Windows Server 2019 with Containers Azure Template. I just added this machine to our Azure AD.
All Windows Updates are installed.

Have you tried to remove the memorylimit parameter?
Have you tried the simplest new-navcontainer without any of the special parameters?

Both commands will work without memoryLimit when I do not include the --security-opt parameter.

So you say without security-opt - it works both with and without memorylimit parameter?
or you say your need to remove both?

Without --security-opt both commands. (docker run, New-NavContainer) will create a running container with or without the memoryLimit.

Could you run a plain windowsservercore with the memory limit and then do
(Get-ComputerInfo).CsTotalPhysicalMemory

Both with and without security-opt, so
docker run -m 3G -ti --rm mcr.microsoft.com/windows/servercore:1809 powershell

and then the same with your security opt

If you start a WindowsServerCore with the securityopt parameter
docker run -it --security-opt "credentialspec=file://containerhost.json" mcr.microsoft.com/windows/servercore:ltsc2019 PowerShell
and then run
Get-ComputerInfo
in the PowerShell prompt - what does that return?

Thats the trick ...

without security-opt
docker run -m 3G -ti --rm mcr.microsoft.com/windows/servercore:1809 powershell

Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

PS C:\> (Get-ComputerInfo).CsTotalPhysicalMemory
34359267328
PS C:\>

With Security-Opt
docker run -m 3G -ti --rm --security-opt "credentialspec=file://containerhost.json" mcr.microsoft.com/windows/servercore:1809 powershell

Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

PS C:\> (Get-ComputerInfo).CsTotalPhysicalMemory
PS C:\> (Get-ComputerInfo).CsTotalPhysicalMemory
PS C:\> nltest /parentdomain
mycgkdom.at. (1)
The command completed successfully
PS C:\>

as you can see, there is no memory info when the security-opt option is included.

In the container what does whoami return?
what does get-computerinfo return?
and what if you run this:
$currentPrincipal = New-Object Security.Principal.WindowsPrincipal([Security.Principal.WindowsIdentity]::GetCurrent())
$currentPrincipal.IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)

PS C:\> docker run -m 3G -ti --rm --security-opt "credentialspec=file://containerhost.json" mcr.microsoft.com/windows/servercore:1809 powershell
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

PS C:\> whoami
user manager\containeradministrator
PS C:\> nltest /parentdomain
mycgkdom.at.  All rights reserved.

PS C:\> (Get-ComputerInfo).CsTotalPhysicalMemory
51530809344

@PleachiM Is the gMSA part of the admin group on the host?

@tfenster No the account was not included in admin group. I added the account and same result after running a new container with security-opt. So no memory information could be found. (Also after Restart of the host)

@tfenster as I know you from the docker workshop on navtechdays I also see no problem to grant you access to our server.

Thanks for that, but I actually have to admit that I wouldn't know what else to check or try... I just want to give it a try in our environment as I know it has worked with previous 2019 releases to find out if it is a problem with 2019 or with Azure AD (or both)

Ok, just one thing to mention here. We do not have an Azure Domain Server or local Domain Server. We are only using the Azure Active Directory Domain Services. As I told you this also makes Problems when joining a machine to this an running a local docker with windows auth. Because there is no way to use the Azure Domain User here.
This server is joined to Azure Active Directory Domain Services like it would be in onprem environments.

That might indeed be the problem here. So let me check (hopefully soon) if it works with our local AD so when can narrow it down. I also am at DockerCon this week and plan to attend a session about win auth in Docker containers done by MSFT together with Docker, maybe AAD will be a part of that as well or I can ask a question about it

at the moment I'm checking this one ...
https://github.com/PowerShell/PowerShell/issues/3080

(Get-CimInstance win32_computersystem).TotalPhysicalMemory
this works

I will have to check that this works with process and hyperv isolation and on server 2016, 2019 and 1709 and 1803. If this works - it might seem like a safer (and faster) bet.
Thanks for the investigation.
Unfortunately, you cannot override the start.ps1 script (as the only one) - maybe I should add that so that nobody gets stuck like this.

I was just investigation the script order for overriding and already thought the same about making this overrideable :)
Thank you all for your fast replies on this issue.

Exactly - building the generic image is fairly easy - everything is on github.
and then run the image with an extract of another container is also fairly easy.

Add -updateHosts to your new-navcontainer - then you will automatically get an entry in the hosts file with your container and its ip address.
Thanks for testing this - I will include this in the next generic image.

I see now that you are just adding a layer to the specific image - nicely done for the test Tobias:-)

I just tested in my own environment (finally set up an Azure AD...) and have the same result: The old way for the memory check fails, the new way works. And gMSAs also work :)
Blog post to follow... @PleachiM Thanks for bringing this up, it finally pushed me to set up this for myself...

If anyone wants to try this, here is the quick walkthrough: https://www.axians-infoma.com/techblog/setting-up-windows-authentication-with-containers-connect-to-azure-ad-domain-services/

This is very cool.
I will update the generic image today or tomorrow and rebuild over the weekend.
NavContainerHelper will also use Get-CimInstance - much faster on Windows 10

Thank you we all appreciate your work. Can't wait to test this next week (or whenever you are ready) and to close this issue
👍

Generic <IP_ADDRESS> has been created and the latest insider builds are using this.
examples:
bcinsider.azurecr.io/bcsandbox:us-ltsc2019
and
bcinsider.azurecr.io/bcsandbox-master:us-ltsc2019
Please try them out.
Another round of full rebuilds is scheduled to next week.

http://aka.ms/readytogo - select the tooling tab

New CU's for NAV 2018, 2017 and 2016 have been created with this fix.


Make the buttons only appear in Creative

Just leaving this here.

maybe by hooking the local player if its in creative? or idk

yay!


How can we help?
I would like someone else to build the manifest.
Please read and ensure the following

[X] The installer meets the above requirements

Please provide the following information
Download Page Url: https://github.com/NationalSecurityAgency/ghidra/releases
Publisher: National Security Agency
Package Name: Ghidra
Description: A software reverse engineering (SRE) suite of tools developed by NSA's Research Directorate in support of the Cybersecurity mission.
Package Version: 10.3.1
Installer URL: https://github.com/NationalSecurityAgency/ghidra/releases/download/Ghidra_10.3.1_build/ghidra_10.3.1_PUBLIC_20230614.zip

The "installer" is a .bat script which goes against the policy of no scripts.
[Policy] Blocking-Issue

As far as I know, it is not an installer, but a launcher. Ghidra does not have an installer, it acts as a portable app.

As far as I know, it is not an installer, but a launcher. Ghidra does not have an installer, it acts as a portable app.

Either way, it is still a bat script and not an executable. It is one of the core principles of WinGet not to allow scripts during install (which includes portable app installs). You can read more about it in the Scripts discussion

I have attempted to add this into winget in the past

Not sure if this helps get it added but the "installation" of ghidra is just downloading the latest .zip and unzipping it to a folder. There is no install.exe or installer script that needs to be ran.
Winget would just need to get zip -> unzip to program files or something and that would be considered installing ghidra

The "installation" of deno is the same way, so there's precedent for non-installer tools. Specifically, it downloads a ZIP file that contains a single file, then extracts it:
https://github.com/microsoft/winget-pkgs/blob/1ef044878b41819fac8efdee86d4d30c84253857/manifests/d/DenoLand/Deno/1.42.4/DenoLand.Deno.installer.yaml#L4-L16
The purpose of adding Ghidra would be to allow winget to manage updates for the user.


Where did the Multiple Configuration option go?

This guide says to use the "options" tab to enable the MultiConfiguration configuration variable, but there is no options tab? The task setting doesn't have anything related to MultiConfiguration either.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 29f86c97-f534-50ad-bfb1-a2cf78c7af94
Version Independent ID: acea40e3-512b-4d9a-9389-71b8b24327be
Content: Visual Studio Build build and release task - Azure Pipelines
Content Source: docs/pipelines/tasks/build/visual-studio-build.md
Product: devops
Technology: devops-cicd
GitHub Login: @vijayma
Microsoft Alias: vijayma

The documentation is out of date. The Multi-configuration option is now on phase settings in the pipeline tasks.


Can I use without cordova?

I am developing a web app and I hope to implement a qr scanner. I came across this library and it looks very robust. I installed it via npm and tried to require('cordova-plugin-qrscanner') as I have done with my other npm modules. I am kind of confused as to how this library can be integrated into my App or if its even possible. I am not using any front end frameworks. Am I completely missing something here? Any clarification would be great. Thanks!

Yes, you should be able to simply include the minified library on the page, and the proper methods will be added to the window object. It's distributed as in UMD format, so it should also work as expected with your CommonJS project.
Be sure to check out the Browser readme section platform for specific notes.

On another note – if you're trying to require it, your CommonJS loader may be looking inside package.json for a main field, which I'm just realizing isn't defined. Looking into that now.

Ok – I just added the property in this commit: https://github.com/bitpay/cordova-plugin-qrscanner/commit/955e375c7d9766a106a6b6f65c0f204fe430fc20
It should be included in the next release of cordova-plugin-qrscanner. Until then, you can get it working by adding the property to the package.json file in node_modules/cordova-plugin-qrscanner like in the commit:
...
"main": "dist/cordova-plugin-qrscanner-lib.min.js",
...

Please let me know if that solves the issue!

You beautiful, beautiful unicorn man. Seems to have worked like a charm This is going to work wonders in my app. This is by far the most robust and well maintained qr scanning library I have found. I appreciate the help and speedy response

Glad to hear it, thanks for opening the issue!


Mind expiry date when looking for sig for subkey

When parsing keys, take the signature with furthest expiration date,
instead of the first signature that comes along.
Fixes: https://github.com/keybase/keybase-issues/issues/2604
Internal ticket: CORE-3879
@oconnor663 Please review, thank you!

But can also wait for @maxtaco , no problem!

Great work! 👍

Thanks a lot!
However: If the signature you find isn't even valid currently, you still have a problem.

I think the signatures are checked beforehand (err = e.PrimaryKey.VerifyKeySignature(subKey.PublicKey, sig), line 594) so if it happens that there is a signature that extends further in time but is invalid, it will not "win".

Also thank you for your patience! It will probably take a few days before client's dependency is updated and a new version is released.


Migration of Bazel dependency system to Bzlmod.

Bzlmod is the new dependency system of Bazel. We need to migrate the current system (i.e. WORKSPACE) to Bzlmod since the current system will be deprecated.
https://bazel.build/external/overview#bzlmod
Once we complete the migration, we will remove the current system with WORKSPACE soon. This should not change anything for users, unless you use Mozc as a third party library in your Bazel WORKSPACE rules.
Please let us know if you have concerns about our migration plan.

We have completed the migration from WORKSPACE to Bzlmod.
We will remove WORKSPACE.bazel in future.
Note,

Bazel 8 will disable WORKSPACE by default.
Bazel 9 will remove WORKSPACE.

https://bazel.build/about/roadmap


Unable to delete old credentials (pre W3C/Credo 0.5)

Screenshots

Description of problem
When trying to remove credentials from the wallet, BC Wallet raises an error.
On the first attempt to remove the credential, the error raised says it is an unsupported credential record type - requires w3c.
On subsequent attempts, the error says the credential exchange record does not exist.
Might be related to https://github.com/bcgov/bc-wallet-mobile/issues/1951 and the introduction of Credo 0.5 as a dependency.
Expected behavior
Credentials can be removed from the wallet without issues.
Steps to reproduce

Open BC Wallet
Receive one or more credentials
Attempt to delete one of the credentials

Environment

[x] Occurs on Android
[ ] Occurs on iOS - unsure, don;t have a device to test with.

Build #: 1.0.17 (1676)
Android Device Model: Pixel 6 Pro

Fixed, tested with 1.0.16 -> 1.0.16 1740 with Showcase lawyer credentials issued in 1.0.16.


Custom error handler

Hi,
let's say I throw an error in the route handler:
const handler = () => async (ctx, next) => {
  try {
    // do something
    await next()
  } catch (err) {
    ctx.throw(500, err)
  }
}

and handle the error with a global middleware:
const errorHandler = () => async (ctx, next) => {
  try {
    await next()
  } catch (err) {
    ctx.status = err.status
    ctx.message = 'error message'
    ctx.body = 'error body'
  }
}

Pino would not log original ctx.throw and log INFO request complete with req and res objects, but no error itself.
Q:

how do I add error to this event?
alternatively, if I log the error myself with ctx.logger.error, how to enrich it with other meta data i.e. timestamp, context etc.?

thank you!
UPDATE to (2)
Apparently, if logged ctx.logger.error({ err }) it would add req and more meta and context around an error.
Was it documented somewhere that I missed and is worth documenting?
Would it be the "proper" way to log errors in custom error handler?

@kraiovsky Did you find any workaround to this?


Tests improvement

This PR improves some test aspects, particularly:

Fixed exception throwing testing.
Increased usage of Spine test utils.
Fixed wrong command handling.

@armiol PTAL.

@armiol PTAL.


Created a hello microverse repository

Review of my repo

Created a hello microverse project with HTML&CSS
Added Linters

Hello @IngDavidTM I have changed my PR title but I am unable to change its description. Is there a way to do it or would I have to make a whole new PR?


Handlebars.js can't apply css/image/js in site?

Issue Summary
I build a single page site; I only need information like name post, content, and image of this post.
In document of HandleBar.js:

Handlebars templates look like regular HTML, with embedded handlebars expressions.

So, I think it will understand and render statics code in *.html file.
I was using handlebars.js into easy manipulate.
I also have the template of a website is HTML file.
I tried to add all code from *.html file to *.hbs file.
You can see like:
<!DOCTYPE html>

	<meta charset="utf-8">
    <title>Custom</title>

	<link href="css/bootstrap.min.css" rel="stylesheet">

    <link rel="stylesheet" type="text/css" href="css/style.css">

Home

About

In this code, will show like:

It only shows only HTML code, seem CSS/JS/(link to an image) not working.
Additional info

Postleaf version: latest.

I'm not sure exactly what you're asking, but a couple things that might help:

Postleaf will only look at .hbs files in the theme directory. It will not look at ..html files.

References to scripts and stylesheets should look like this:

<link href="{{theme_url 'css/bootstrap.min.css'}}" rel="stylesheet">
<link rel="stylesheet" type="text/css" href="{{theme_url 'css/style.css'}}">

This will ensure that the stylesheet will always be linked properly, even if Postleaf is running from a subdirectory.

I hope my answer above helped. If not, please ask your question with more detail on the forum so we can better help. I don't believe this is a problem with the software and I'm trying to reserve GitHub issues for bugs. 🙂 🐞

Thank you very much @claviska


SUBMARINE-893. Add submarine-operator e2e test case for create/delete submarine

What is this PR for?

Add submarine-operator e2e test case TestSubmitSubmarineCustomResourceYaml:

Create a test namespace
Create a submarine custom resource
Wait for the submarine to be in RUNNING state
Delete the submarine custom resource
Delete the test namespace

What type of PR is it?
[Feature]
Todos
None
What is the Jira issue?

https://issues.apache.org/jira/browse/SUBMARINE-893
How should this be tested?

# Add -v to see additional log
# Add -count=1 to run the next time, otherwise, go will use the cache result.
go test ./test/e2e

Screenshots (if appropriate)
https://user-images.githubusercontent.com/17617373/131078695-a00b936c-b81a-4ced-a784-961d9ecdbf0d.mov
Questions:

Do the license files need updating? No
Are there breaking changes for older versions? No
Does this need new documentation? No

@MortalHappiness @kevin85421 Can you help me review this PR ?


it will support docx or not?

I want to preview microsoft word by using viewerJs and currently it is not working, is there any solution to make it?

The only solution would be to convert from DOCX to ODT on the server. You can use LibreOffice for this.


🌱 Upgraded to go 1.17

Please check if the PR fulfills these requirements

[ ] Tests for the changes have been added (for bug fixes / features)
[x] PR title follows the guidelines defined in  https://github.com/ossf/scorecard/blob/main/CONTRIBUTING.md#pr-process

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
feature

What is the current behavior? (You can also link to an open issue here)
fixes #929

What is the new behavior (if this is a feature change)?
Upgraded to go 1.17

Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)
no

Other information:

/ok-to-test sha=064013e


Looping through list of QuantumCircuits to get result from individual circuits

Information

Qiskit Terra version:  0.16.0
Python version: Python 3.7.3
Operating system: Mac OS X
ProductVersion: 10.15.1
BuildVersion: 19B88

What is the current behavior?
Given a results object that comes from the execution of a list of quantum circuits, the retrieval of the results for the different circuits does not work for
for qc in qc_list.
Steps to reproduce the problem
import csv
import numpy as np

from qiskit import Aer, execute
from qiskit import QuantumRegister, ClassicalRegister, QuantumCircuit
from qiskit.circuit.library import ZZFeatureMap, RealAmplitudes

backend = Aer.get_backend('qasm_simulator')

n = 4

blocks = 1

circuit = QuantumCircuit(n)
feature_map = ZZFeatureMap(n, reps=2, entanglement='linear')
var_form = RealAmplitudes(n, reps=blocks, entanglement='linear')
circuit = feature_map.combine(var_form)
circuit.measure_all()

x_list = [[0.80, 0.55, 0.22, 0.035], [0.83, 0.51, 0.24, 0.03], [0.81, 0.55, 0.22, 0.03]]
params = np.random.rand(len(var_form.ordered_parameters))
qc_list = []
for i, x in enumerate(x_list):
    parameters = {}
    for i, p in enumerate(feature_map.ordered_parameters):
        parameters[p] = x[i]
    for i, p in enumerate(var_form.ordered_parameters):
        parameters[p] = params[i]
    qc_list +=[circuit.assign_parameters(parameters)]

results = execute(qc_list, backend).result()
probs = []
# works
# for j in range(len(qc_list)):
#     counts = results.get_counts(experiment=qc_list[j])
# doesn't fetch the correct circuit
for qc in qc_list:
    counts = results.get_counts(experiment=qc)
    print(counts)

What is the expected behavior?
Fetch the results corresponding to the individual quantum circuits.
Suggested solutions

This issue arises from assign_parameters returning QuantumCircuit instances with the same name. There is an open (but stalled) PR with a possible fix https://github.com/Qiskit/qiskit-terra/pull/3775 .

Thanks I see.