reflect: back out multiple keys in key:value pair in struct tag

Hello 👋
I'm actually boring to set the same value for multiple key in my struct tag. Maybe I missed something but here is the boring thing:
type MyStruct struct {
  Field1 string `json:"field_1,omitempty" bson:"field_1,omitempty" xml:"field_1,omitempty" form:"field_1,omitempty" other:"value"`
}

What I'm proposing is:
type MyStruct struct {
  Field1 string `json,bson,xml,form:"field_1,omitempty" other:"value"`
}

Thanks 👍

Just noting that this would mean that struct tag keys cannot contain a comma character.  The current documentation at https://golang.org/pkg/reflect/#StructTag is "Each key is a non-empty string consisting of non-control characters other than space (U+0020 ' '), quote (U+0022 '"'), and colon (U+003A ':')."  So this would be a change.  Of course, it's fairly unlikely that anybody is using a tag key with a comma.

It could be a breaking change of course. But I think it could be a great new feature for Structure Tags :)

I have a large corpus of Go code that I can scan at some point to see if anyone uses , in tag names in practice.
Will try to do that before next week.

What if we use a space-separated list instead of a comma-separated one?
type MyStruct struct {
  Field1 string `json bson xml form:"field_1,omitempty" other:"value"`
}

The spaces form is more analogous to what happens in Go argument lists:
in x int, y int, z int, you can remove the repeated  int, leaving x, y, z int.
If we do the same here, then
in json:"x" bson:"x" xml:"x", you can remove the repeated :"x", leaving json bson xml:"x".
If we inserted commas during this step, you'd need to explain where they came from
(they weren't there before!).
The space form is also quite a bit less of a wall of text and thus easier to skim. Compare:
Field1 string json bson xml form:"field_1,omitempty" other:"value"
Field1 string json,bson,xml,form:"field_1,omitempty" other:"value"
And unlike comma, spaces cannot appear in valid tags today,
so we would not be redefining technically valid syntax to mean
something slightly different.

Space-separated is much better and it indeed looks like Go argument list.
Regarding proposal: I haven't use more than 1 file tag (json or yaml) before, but last week have played with different config files and leaving all this verbose repetitions (field1 for JSON, also for YAML, also for TOML, also for ...) will make multi-tags much easier to use.

I must admit that space separated could be a good idea which won’t break anything. Personally I feel it less comprehensible than comma separated and in go we have comma between arguments (we don’t write a b c int but a, b, c int so comma is more go in fact lol) but a choice have to be made and don’t belongs to me :)

OK, so with the syntax I suggested last Thursday (space-separated instead of comma-separated keys), there is no concern about backwards compatibility. There hasn't been much discussion otherwise.
I did make some progress analyzing tags in my Go corpus last week, and I found that 0.06% of tags have commas in keys (outside quoted values). All of those are mistakes. So we could plausibly use comma and not break things, but the space is more in keeping with the current syntax anyway.
Based on the discussion above, this seems like a likely accept (with spaces, not commas).

No change in consensus, so accepted.

Hi! I would like to work on that feature. Is it possible? :)

Sure, go ahead.

Here my change: https://go-review.googlesource.com/c/go/+/248341

Hello @bynov . Are you still working on this?

Hi, @deven96
I have already sent PR on Gerrit. It's in review now :)

Lol... darn it
Goodluck mate @bynov

Hello, into which version of go this feature will be available ?

@scorsi probably Feb 2021, when Go 1.16 will be released.

I'm trying to add this new syntax to one of my tools gomodifytags which is used in many editors (i.e: VSCode, Vim, etc..). It allows one to add or remove struct tags in an easy way. You can find more details here: https://github.com/fatih/gomodifytags
One particular edge case I've found is, assuming the user wants to add a new tag to an existing field for the same value.
type T struct {
	Server string `json:"server"`
}

Adding the new key in the old format would produce:
type T struct {
	Server string `json:"server" xml:"server"`
}

But in the new format, the tool need to output it in:
type T struct {
	Server string `json xml:"server"`
}

So the problem I see here is, how should I, as a tool owner know which format is the correct one for the user? Outputting in the new format would mean that we should assume the application is built with the latest (i.e: 1.16) Go version. But if they run it in an older version, the new format won't work and it'll break their applications.
One can say that outputting in the old format should work and hence the tool always should output in the old format for backwards compatibility. If I choose this approach, then some other use cases arise. Assume the user already uses the new format:
type T struct {
	Server string `json xml:"server"`
}

If the user adds a new tag, because the tool outputs in the old format, suddenly it changes to something that the user didn't intended to use:
type T struct {
	Server string `json:"server" xml:"server" bson:"server"`
}

I think this is the only approach I can use for now. But then, I wonder how I can make it easier for the user to use this new syntax. I feel like there is a compromise I need to consider and whether this change would cause disruptions in the future because not a lot of services might run with the old versions. It looks like a ticking bomb waiting to explode and I hope we can reconsider the syntax due the reasons listed here.

@fatih Isn't this what the go language version in the go.mod file is for? Can't you just use the new style if version ≥ 1.16, and otherwise stick with the old one?

@uluyol I used to think the same thing but I don't believe that's true. A go.mod file declaring 1.16 can still be compiled by 1.15 or lower.

If the go.mod files says go 1.16, that means that the files in the module may use constructs introduced in language version 1.16.  A person using a Go 1.15 or earlier compiler can still attempt to compile the module.  That is not prohibited.  But if an error occurs while compiling, the go tool will report "note: module requires Go 1.16".
So in general it is OK for a transformation tool to use Go 1.16 language constructs if go 1.16 appears in the go.mod file.

I fully agree with @ianlancetaylor; a tool should use the go.mod language version for this decision. In this case it's slightly trickier, as an older version of Go would fail only when using vet/test to parse the struct field tag, and not at compile time, as I'm pretty sure struct field tags are not fully parsed at compile time.
But I still think that is okay. If a module declares Go 1.16 as the language version, and a downstream user insists on using Go 1.14 or 1.15, I argue it's up to them to be extra careful that their software still works even after it builds. A Go program/module building on an older version of Go does not mean that it will work fine. Fully vetting/testing it will the older Go version is the only way to be sure.

I argue it's up to them to be extra careful that their software still works even after it builds

I should clarify, however, that I think this might end up being a problem with people installing Go programs via go get/install following README instructions. Those people are likely not Go experts, and they might be using the Go version that came with their system, often an older version. If their program install works, they would have no reason to think that the program might not work properly.
That said, I still think that is a problem with Go builds in general; this change just adds another way to end up in this weird spot.

But if an error occurs while compiling, the go tool will report "note: module requires Go 1.16".

This heuristic works well for language changes that break existing syntax, but not so well for changes like this one that change behaviour but don't result in errors when using an old compiler version.
For example, say some third party package changes to use this feature, and I (still using go 1.15) update to use the new version. My code is now broken with no indication of the fact. go vet would pick this up, but it's uncommon for people to run go vet on third-party code (doing so very often spews out a raft of false positives).
When using an older Go compiler, it's common to import modules that declare newer Go versions, because any newly created module declares the current compiler version even if isn't using new language features, so seeing that happen isn't an instant red flag (aside: go list -f '{{.Path}} {{.GoVersion}}' -m all isn't hard to type, but isn't obvious either, if you want to check this).
I worry that in larger ecosystems, this change might end up with significant non-obvious brokenness that only emerges after things break in production.
I don't currently see an easy way around it.

This heuristic works well for language changes that break existing syntax, but not so well for changes like this one that change behaviour but don't result in errors when using an old compiler version.

Yes.  This is discussed in detail at https://go.googlesource.com/proposal/+/refs/heads/master/design/28221-go2-transitions.md .
This isn't quite a redefinition, as the new code generates a runtime error with pre-1.16 versions of the standard library, but you're right that it's problematic in that it doesn't generate a compile time error.
So I think I agree with you: although as I said above using the new syntax is in some sense permitted for Go 1.16 or later, full safety suggests not using the new syntax until Go 1.15 and earlier are no longer supported.

This isn't quite a redefinition, as the new code generates a runtime error with pre-1.16 versions of the standard library

I don't see a runtime error when running this code under Go 1.15,
so it seems like it actually is a redefinition.

You're right.  I was mistaken.

@ianlancetaylor just to sort out any confusion, is the decision here to include this change in Go 1.16 even though it indeed is a redefinition?
As I read https://go.googlesource.com/proposal/+/master/design/28221-go2-transitions.md#language-redefinitions it looks like redefinitions should be avoided:

I think the only feasible safe approach is to not permit language redefinitions.

I do understand that there might be exceptions for critical changes, but do this feature really falls into that category?

To be clear, the redefinition is that this program prints an empty string in older versions of Go (and is rejected by go vet in those earlier versions):
package main

import (
	"fmt"
	"reflect"
)

type T struct {
	Server string `json xml bson:"other"`
}

func main() {
	var t T
	fmt.Printf("json tag: %q\n", reflect.TypeOf(t).Field(0).Tag.Get("json"))
}

And now in a newer version, it will print json tag: "other".
So the unexpected behavior is that if people adopt this syntax when using Go 1.16, but then they publish a library and Go 1.15 users use it, then it won't work the same in the earlier versions, which will be blind to the tag.
We are not redefining existing programs (no existing programs use spaces in tag name syntax and pass go vet).
We are however defining a new syntax that will behave silently differently in older versions of Go.
The question is whether that's OK.

It seems like there are three paths forward.
(1) Kill this feature forever.
(2) Keep the feature as is, with the rocky transition.
And (3) do a more staged transition like the //go:build lines. That staged transition looks like:

In Go 1.16, we roll back the current changes and only put code in the compiler to reject struct tags with spaces in the keys, with a note saying "invalid use of Go 1.17 struct tag syntax". (This would be a big step for the compiler but any code rejected would not be passing go test's implicit go vet today.)
In Go 1.17, we take out the compiler check and put back the current CLs.

Then if someone adopts the new syntax ahead of their importers adopting a version of Go that understands it, the importers will get a clear failure message instead of silently different behavior.
It does seem like maybe (2) is a bit too subtle for the case where you're depending on someone else's code and they've updated more quickly than you realized (or you've updated more slowly than they realized). So maybe we should do (1) or (3).
Thoughts?

Reopening to discuss transition.

At least based on the discussion in #tools on Slack, 2 has seemed problematic to many, because it's subtle and "silent".
In terms of 1 vs 3, the question I had is what were the arguments in favour of this in the first place? i.e. is there a large corpus of code that would be demonstrably improved by this change, in such a way as to outweigh the cost of implementation/transition?

I vote for 3, but there's still a problem for 1.15 users if a library upgrades directly from 1.15 to 1.17.
It definitely narrows the window though.

I can't see how this change is worth doing (3) so I would say (1) unless someone brings additional data.

I'd vote option 1 as well.
I don't believe the cost (breakage in tools, upgrade paths, and teaching newcomers about two different syntaxes inside struct tags) is worth the benefits.
And with tools like Fatih's gomodifytags a lot of this can be automated anyway.

I personally vote for using ",", makes it more explicit and easier to read, otherwise #3.

Why if instead of space we use coma as I suggested at the beginning ? Does it will fix that transition feature ?
I understand the issue but killing that feature forever is a little bit too disproportionate no ? There is always drawbacks but we have to look into benefits and here there are. Just to sum one very useful: duplicating code is bad and error prone, I think we are all agreed, when creating structs for web app communicating in different formats and to different DB, you are duplicating a lot those struct tags : json, bson, xml, csv, pg, ...

Does it will fix that transition feature ?

Unfortunately not, it would be a slightly worse situation since go vet won't detect the invalid tag in Go <= 1.15.

no existing programs use spaces in tag name syntax and pass go vet

FWIW, I argued in #3059 that fixing a cmd/compile type-checking mistake should be safe because the affected programs already failed go vet, and @ianlancetaylor and @randall77 disagreed and argue the fix needs to be gated by -lang.
In that issue, go/types (and thus go vet) already rejects programs that never use a variable except for assignments, but cmd/compile mistakenly treats variables as used if they're captured by reference by a function literal. Notably, in the case of #3059, the go/types type-checking failure cascades and affects downstream go vet / go test usage too; see https://github.com/golang/go/issues/3059#issuecomment-674334132 for an example where example.com/a exercises the cmd/compile issue, and it causes go vet example.com/b and go test example.com/b to fail.
I think we should be consistent about whether "passes go vet" affects the applicability of Go 1 compatibility.
--
I'll also say I think it's an error that the Go 1.N toolchain will silently accept modules that specify Go 1.(N+1). That precludes us from using -lang to redefine existing syntax except through the whole staged transition process, which just delays rolling out new features to end users.
E.g., if Go 1.15 didn't silently compile packages that declare go 1.16 in their go.mod, then one path forward here would be to make the compiler responsible for expanding `foo bar:"xyzzy"` into `foo:"xyzzy" bar:"xyzzy"` when -lang=go1.16 is specified. (Maybe that's still not desirable for some reason; e.g., how should this affect type identity?) Existing packages that might use spaces would continue to behave the same as always.
But we can't do that in Go 1.16, because the Go 1.15 toolchain will be supported for another 6 months. And even after that, some users are likely to continue using older, unsupported Go toolchains, and will risk silently miscompiling any packages that start using this feature after Go 1.17.

colon instead of space would also work: `json:bson:"abc"`
like in a chained assignment.

I'll also say I think it's an error that the Go 1.N toolchain will silently accept modules that specify Go 1.(N+1). That precludes us from using -lang to redefine existing syntax except through the whole staged transition process, which just delays rolling out new features to end users.

Noted, and I agreed with your position at first, but @ianlancetaylor convinced me otherwise when we made the decision.
Because we make the go version get set automatically during "go mod init" (which is good!), many packages overstate their strict Go version requirements, and unlike normal package requirements, we can't just push the build to the newer version of Go automatically.  This is a pretty likely situation and it would harm the ecosystem.
The way to avoid the eager rejections is to commit that we will not silently change the meaning of an existing program. We can redefine existing building programs to non-building programs, which won't affect older versions. And we can redefine existing non-building programs to become building programs, which will fail to compile on older versions and then trigger the "note: module requires Go 1.N". But we cannot redefine existing building programs to have a different meaning. We've committed to that restriction.
That is, we have taken "using -lang to redefine existing syntax" - in the sense of changing a program from one working meaning to another - off the table entirely.
#3059 was not "redefine existing syntax" though. It was "break working program", which is exactly what the Go version in the module file (passed to the -lang flag) is for. So that came out right.

The discussion here seems to be leaning toward (1) remove the feature entirely for Go 1.16.
The rationale is that it's a fairly minor improvement, we can't ship it compatibly in Go 1.16, and it doesn't really seem worth the pain of a multistep transition.

Could it support multiline? Like this:
type MyStruct struct {
  Field1 string `json:"field_1,omitempty"`
                `bson:"field_1,omitempty"`
                `xml:"field_1,omitempty"`
                `form:"field_1,omitempty"`
                `other:"value"`
}

@JiamingLinn See #15893, #38641, #42182.

that's really ugly my friend @JiamingLinn

@batara666 please be respectful in your comments as per CoC: https://golang.org/conduct. Saying that something is "ugly" really doesn't add anything constructive.

Based on the discussion above, this seems like a likely accept.

Sorry for confusion - by accept I really meant roll back. Will update header.

type MyStruct struct {
Field1 string json bson xml form:"field_1,omitempty" other:"value"
}

Alright, we accept this version xoxo 🎉🎉🎉🎉🎉

No change in consensus, so accepted.
Meaning: we have accepted the suggestion to roll back this new feature.
(Sorry again, this second discussion should have been a separate issue.)

No change in consensus, so accepted.
Meaning: we have accepted the suggestion to roll back this new feature.
(Sorry again, this second discussion should have been a separate issue.)

Everything is backed out now.

Everything is backed out now.

The idea is good but how to make it work with json.Marshal for exemple ? Because it uses the internal tag library. :)


Sprite tool should always ignore sprite cache

Or there should be an option for which one you load up. The sprite tool should always give you the source file to preview since the sprite builder should be built against that.

This is fixed. The sprite tool should always load up exactly what the game will see.


[bug] format tool: the index signatures in schema stmt are always formatted before attributes

Bug Report
Please answer these questions before submitting your issue. Thanks!
1. Minimal reproduce step (Required)

create main.k with content:

schema A:
    name: str
    [other: str]: str

run kcl-fmt main.k

2. What did you expect to see? (Required)
schema A:
    [other: str]: str
    name: str

3. What did you see instead (Required)
schema A:
    name: str
    [other: str]: str

4. What is your KCL components version? (Required)

cause:
The format tool works by printing the ast as the formatted code. And the schema stmt walker always first walk the index signature nodes, then walk the schema body node(which contains the attributes)

Since the AST is designed to not include the index signatures in the schema body, the order between the index signatures and attributes is not recorded.
@Peefy Is this the AST is designed to not include the index signatures in the schema body designed in purpose? or could somebody explain why?
cc @NeverRaR

@amyXia1994 In a schema, the syntax of index signatures is not close to the syntax of schema attribute definitions, and the order is independent in the schema. We need to define index signatures in the specification as being written as early as possible because they represent the overall composition of schema attributes.

Ok, got it, so the behavior of described in this issue is actually a feature of the code style spec. Let's add new rules to clarify this.

tracked in https://github.com/kcl-lang/kcl-lang.io/issues/183


Remove wrong "(default)" from Configurations.md

blank_lines_upper_bound has default value 1, not 2.

Thanks!


Update unlock-encrypted-linux-disk-offline-repair.md

Merge my changes with the original document.
I had a discussion with the technical auther Didier Ambrose and he validated my proposed changes and confirmed to add the below changes:
I have created a pull request as per the confirmed proposed changes.
Change 1-------------
In Azure, disk encryption is supported for Microsoft endorsed Linux VMs. There are few basic requirements to enable Azure disk encryption in Linux VMs like :
•	Azure Key Vault
•	CLI or Powershell cmdlets
•	Device-Mapper DM Crypt
DM Crypt
Device-mapper is infrastructure in the linux kernel that provides a generic way to create virtual layers of block devices. Device-mapper crypt target provides transparent encryption of block devices using the kernel crypto API. To gather more information about the functioning of DM-Crypt , please refer https://gitlab.com/cryptsetup/cryptsetup/-/wikis/DMCrypt
Change 2------------
There could be multiple LinuxPassPhraseFileName if there are more than one disk in the VM which are encrypted. The LinuxPassPhraseFileName will be enumerated in accordance with the number of disks attached with the VM in the similar order as their LUN numbers.

@radawn17 : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@AmandaAZ : Thanks for your contribution! The author(s) have been notified to review your proposed change.


get_replay is not working

Calls to get_replay result in
[
  [
    "hexstring",
    1,
    "2023/07/10 06:36:18",
    "0.2.1",
    "0.0.2",
    "0.0.2",
    "",
    ""
  ],
  [
    3
  ]
]

I've seen 3 and 4 in that last part instead of the replay data. Which I'm guessing is an error.
I checked a real request to get_replay and there's no change, so I tried using a real token, that worked.
Looks like auth_ticket has changed.
Before the auth_ticket was:
<auth_ticket> structure:
    <game_token_length>
    <game_token>
    <session_size> (always equals 24)
    <unknown_1> (always equals 1)
    <unknown_2> (always equals 2)
    <ip_address> (obfuscated ip address, I suppose just <auth_ticket> identifier)
    <unknown_3> (random number, but 0 working for me)
    <connection_time> (time of session in milliseconds)
    <connection_count>

eg:
14000000 token len
5088EF43300E7A510FF1DD0B0100100162EDB663 token
18000000 24
01000000 1
02000000 2
82EC43A9 IP
7F211FB1 random (we set to 0)
EB546E03 (time of session in milliseconds)
03000000 connection count

now it looks like:
<auth_ticket> structure:
    <game_token_length>
    <game_token>
    <session_size> (always equals 24)
    <unknown_1> (always equals 1)
    <unknown_2> (always equals 5)
    <unknown_3> ?
    <unknown_4> ?
    <connection_time> (might be the same)
    <connection_count> (probably the same)

eg:
14000000 = token length = 20
42E208184E0874EC4F317E00010010014BB9AB64 = token
18000000 = session_size = 24
01000000 = 1
05000000 = 5 (used to be 2, maybe this is a version?)
14D2C326 = ? not my ip
110F0B5E = random?
D951E500 = might still be session time
05000000 = probably still connection count

My IP address does show up later in the app_ticket, and it seems like it never used to.
So I'm guessing they got rid of it from the auth_ticket and replaced it with ... something.

I don't know how helpful this is, but the 14D2C326 sequence translates to <IP_ADDRESS>, which geolocation websites identify as a Microsoft Azure address in Tokyo. Could it one of the game's cloud-hosted services?

@alestef That is interesting, thank you.
I tried using that IP there, plus I tried using 110F0B5E for the random value, then I tried 00000000 for the random value.
All didn't work :(

Hi, I've been looking at all this and figured I should post some relevant links in case anyone else is trying to get it fixed since the dev posted about it on Twitter;
Here is a discussion for Totsugeki where a lot of the underlying mechanics here are discussed: Broken by December Patch?
It looks like people are using proxifier to proxy all (Or just GGST) traffic through to polarproxy which decrypts & writes to a .pcap file for inspection in wireshark. These tools would probably be helpful to someone better versed with Netcode shenanigans, but I'm mainly trying to build a jumping-off point for anyone trying to get this working.
Also, @nemasu and @alestef, thank you both so much for trying to get this figured out; could either of you please elaborate on what & where in the code you're making changes to test these things? I've been having some trouble getting rating-update to run on my machine in the first place, much less changing header contents to fit the updated API.
It looks like the game_token mentioned is token.txt, made by running generate_steam_token.py. This is then used in generate_replay_request as part of the header. Is int1 in the RequestHeader struct the one that has now been changed from (always equals 2) to (always equals 5), or is this not a struct that should even be touched for this issue? Regardless, it doesn't feel like this is the most effective way for me to do testing, are y'all just using the Python code for building/sending a request mentioned here?
Sorry for the long post, any advice would be greatly appreciated.

@collinsjacob127 You should be able to just test using the python code - it has a function both for logging in and generating a token as well as fetching replays. The thing I've done in the past is used the python code to figure out what's going on and then just mirrored the changes in the replay API in the rust code, while still using the token generated from python (rust doesn't have the equivalent steam libraries, so logging in isn't possible)

@halvnykterist Just curious, how did you end up getting it fixed? Browsing the changes it looks like in the struct you updated the user ID from a string to an int but that seems almost too easy? Was that all that changed with get_replay?

RU now uses steamworks, no need to reverse engineer this.


Revamp errors in aws-smithy-types

Motivation and Context
This PR revamps errors in aws-smithy-types to adhere to RFC-0022: Error Context and Compatibility.
This PR is part of a series that will fully implement the RFC, tracked in #1926.
Changelog entries added in #1951.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

The merge conflict when pulling in the latest main in aws_smithy_types::base64 was substantial enough that this file deserves another review: rust-runtime/aws-smithy-types/src/base64.rs


Fixes for updated ManagementPortal

Add a line in install-radar-stack.sh to generate the keystore for JWT signing
Update the OAuth clients details for use with the new authorization mechanisms

@dennyverbeeck I also updated the nginx.conf.template to include the context path of the spring boot app.


Fix loop variable error on LV2export

Location: "Write preset.ttl" section. Index of program was erroneously passed to plugin.getStateHints(), which made LV2export misbehave.

oh nice one. must have been copy&paste lapse on my side. thanks!

My pleasure!


songs do not finish

Hey, this project was highlighted in the Linux Magazine, issue 275. I am giving it a shot on my Macbook. It is pretty great for quick focus music.
One issue is the track lengths usually look like about an hour long, but the tracks always seem to cut off and end early.
maybe its something in the custom impl here? https://github.com/guptarohit/mfp/blob/main/src/mp3_stream_decoder.rs
no PR but just raising the issue I see.

If you have any pointers that could help me reproducing this issue, please do share 🙂 It would be really helpful. Also, which OS you're using?

MacBook M1, osx


Document application changes

Deployed Netlify example here: https://jovial-fermat-4a0934.netlify.com/

forceRefresh set to true on router AppRoute.js <Router location={path} context={{}} forceRefresh={true}>
comment out delete state.viewBag; in server.js
comment out this.updateLanguage(); in RouteHandler
pass SSR state to i18ninit in index.js i18ninit(__JSS_STATE__.sitecore.context.language, __JSS_STATE__.viewBag.dictionary) to disable dictionary call

Yep, that did it! Working my end now 🙌 Thanks @erzr


EC2InstanceMetadata threading bug

Expected Behavior
Amazon.Util.EC2InstanceMetadata exposes static members to retrieve metadata that I would expect to be thread-safe.
Current Behavior
The EC2InstanceMetadata uses a static Dictionary<string,string> to cache instance metadata without implementing any synchronization:
private static Dictionary<string, string> _cache

The Dictionary<TKey, TValue> class is not thread-safe: To allow the collection to be accessed by multiple threads for reading and writing, you must implement your own synchronization.
Possible Solution
Implement synchronization, for example by wrapping all read and write access to the _cache dictionary in a lock statement.
Steps to Reproduce (for bugs)
Difficult to reproduce, but there is the risk of heisenbugs if there is concurrent access from multiple threads.
Context
Low priority / low impact
Your Environment

AWSSDK.Core version used: <IP_ADDRESS>
Operating System and version: Windows 10
Visual Studio version: 2017
Targeted .NET platform: 3.5 and .NET Standard 2.0

We have regular problems with our web applications "hanging" on this issue: When multiple threads try to access EC2InstanceMetaData methods, and they all try to access the _cache Dictionary simultaneously, the dictionary seems to get in a loop, consuming 100% CPU. This makes this class basically unusable in multi-threaded scenarios.
Even besides that, it seems a pretty important issue that public static members in an SDK are not thread-safe.

Fixed in AWS SDK Core <IP_ADDRESS>


Automatic Turn counter randomly stops working

Almost every game I play, at some point during the game, the turn counter stops working.
Clicking through the Phases or right clicking the button doesn't progress the turns, at which point every counter has to be updated manually.
I haven't been able to find what triggers this issue.
Some times it's broken from turn one, other times it breaks half way through. It's very rare for it to work through a full game.
The only thing I've noticed is that every time it has broken, one of the players have used fixed missions. And the fixed mission cards don't get flipped over when secondary's are revealed. They are locked face down, and have to be unlocked and manually turned over.
Thanks!

The face-down-ness is WAD - you're supposed to flip them over yourselves as soon as both players have selected (though I could add some code to automatically flip them as soon as both players have locked them in, I'll likely do that if I get a chance).
Turn counter definitely sounds like a bug, though, thanks! Though it might be tricky for me to find the bug without a repro scenario :(

If the fixed cards staying face down isn't a bug, then that's completely fine. It's just a mild inconvenience to have to unlock, flip and lock again. I just found it strange, which is why I assumed it was a bug :)
I'll continue to play around with it and see if I can narrow down the trigger. But so far, the only thing I've noticed is that I've had Fixed missions in all the games where it bugged out. I'm fairly certain that every game where it worked all the way through, both players had tactical missions. My sample size is pretty small though, so it's probably not very useful information.
If there's anything I can do to help you find the bug, please let me know :)

Okay..
It's not bugged, it's just locked to prevent the opponent from ending the other players turn. "facepalm"
Since my friend keeps forgetting to use it, I've been trying to help him move the counter along, but it obviously doesn't let me end his turn, so we thought it kept bugging out. But in reality, he just needs to start pressing the button :D
Sorry for wasting your time <3

Fixed in 2aeb85ce7cfac7501cdc873d3d98971ecdd5fc3e


Add shell.nix file.

Use nix-shell --pure to open a shell with all dependencies.
Changes all perl invocations to use env. Since the linux kernel splits
shebangs only on the first whitespace, the -w has to be replaced by perl’s
use warnings; pragma.

Nice, thanks


Changes to source code not reflected in served api

I was able to get "make serve" to properly serve up the api, but now when I make a change to https://github.com/ServiceNow/picard/blob/main/seq2seq/serve_seq2seq.py to attempt to add another api call or something of the like, the change isn't reflected in the api served up by make serve.
To remedy this, as I saw suggested in another issue, I forked the repo and committed my change to git with still no difference in the served up api. Will I need to rebuild the "build-eval-image" and push it up to the docker hub every time a change is made? @tscholak

I've determined that this is the case. The author mentions in another issue how "if you modify code, you need to push it to git and then rebuild the docker images. A workaround would be to mount your local copy of the code in the docker container, but that is untested."


[1.13] Author backport of #28928 (Dns proxy use original source address and port)

[ ] #27582 (@tklauser)
[ ] #28928 (@jrajahalme)

Once this PR is merged, a GitHub action will update the labels of these PRs:
 27582 28928

/test-backport-1.13

/test-1.23-4.19

Travis CI has now failed twice on amd64 in the setup (before running anything Cilium related), while an arm64 it succeeded on the 1st try.


Should beam center finder construct inner IofQ pipeline based on outer pipeline?

In  the beam center_finder we repeatedly compute IofQ in quadrants. This is done by constructing a sciline pipeline and calling pipeline.compute(IofQ[...]). This "inner" pipeline is a subgraph of "outer" pipeline.
Right now we construct this subgraph manually inside the beam center finder. I think this is both a bit cumbersome and error-prone. It would be better to take the outer graph, copy it and replace parts of it.
Then we would only need to change MaskedData[SampleRun]], NormWavelengthTerm[SampleRun] and BeamCenter inside the beam center finder routine, instead of manually copying all required providers to the new pipeline.
Encountered this issue when working on #29 . (See this for a bad example of how to solve the problem.) In the PR new parameters are added to the IofQ pipeline, for example DetectorPixelShape[RunType] per run type. These will also need to be part of the inner pipeline. We can add the new providers as requirements to the beam center finder and add those to the inner pipeline, but it doesn't seem practical if every time we add new components to the outer pipeline we need to remember to also add them in the beam center finder.
I'm not sure what the best solution is here. In principle the inner pipeline should depend on the pipeline it is part of, but maybe that is too complicated. For now it's probably best to just add the new providers as requirements to beam center finder. Maybe this should be converted to an issue in sciline instead.

I don't think this is conceptually possible if we want to run the beam-center finder as part of a workflow?
Two brief answers (that may not fully satisfy your concern):

If we drastically change a pipeline introducing new dependencies via updated types and dependencies we will always notice, since the graph in the beam-center finder will fail to build. This is rare (I hope), so I do not see this as a usability issue.
It is not clear if the version of the beam-center finder that uses the IofQ pipeline will actually be used in practice. We have since added a much simpler and faster version which does not need it.

I don't think this is conceptually possible if we want to run the beam-center finder as part of a workflow?

I don't see why it wouldn't be? Conceptually there's nothing particularly unusual about a workflow where one step of the workflow is to run the workflow repeatedly with different parameters until good parameters are found so that those can be used in the real workflow.
I think might be cases when we don't notice differences between the inner and outer version of the pipeline. Possibly there's one minor example of that in the current workflow. In the beam-center-finder pipeline (the "inner" pipeline) we use UncertaintyBroadcastMode.upper_bound  regardless of what is used in the outer pipeline. (Maybe this is intentional here?)


Proof server - readiness test

It'd be great if the proof server tab would also do a test to see if the server is ready.  Yes, it says it's UP but if it's not yet ready to respond to users hitting the API then it's confusing for users.

We do do this in a sense. See https://github.com/FredHutch/shiny-cromwell/blob/d3588110401e72a9add0cd5942d991096eb012a7/app/server.R#L231 where we disable the start button that page if the metadata from the proof API says the server is running (i.e., canJobStart is FALSE).
Are you wanting something in addition that pings the actual cromwell server rather than using metadata from the proof API?

Feels like we can close this one based on Scott's response, plus the "Job status: RUNNING" metadata at the top of the PROOF server tab. Probably more important to prioritize #113


USBPorts.kext incorrect EH01 and EH02 ports

Hi @headkaze
I have a Lenovo ThinkPad X240 which is Haswell and has EH01 Controller where on some devices Touchscreen and in my device it's a EMV smart card reader.
The problem is that the EMV Smart Card Reader is tied to EH01 > PR11 > HP15 port
With USBInjectAll.kext + my custom SSDT-UIAC (SSDT-USB) everything is fine.
However i wanted to get rid of the SSDT-USB and USBInjectAll.kext to have a cleaner setup with just USBPorts.kext
There is a problem with generation of EH01 and EH02 ports with your hackintool.
HP15 doesn't appear on IOREG and just the PR11 hub.
Attached you can find both Ioregs (with USBInjectall+SSDT and with USBPorts.kext) and my SSDT-UIAC (SSDT-USB) so you can check the problem.
USB.zip
CC @vit9696 @Andrey1970AppleLife

I don't support hackintool.
https://github.com/acidanthera/bugtracker/blob/master/DOCUMENTS.md
ACDT0116PRU
https://github.com/acidanthera/OpenCorePkg/blob/master/Docs/AcpiSamples/SSDT-EHCx_OFF.dsl

I don't support hackintool.
https://github.com/acidanthera/bugtracker/blob/master/DOCUMENTS.md
ACDT0116PRU
https://github.com/acidanthera/OpenCorePkg/blob/master/Docs/AcpiSamples/SSDT-EHCx_OFF.dsl

I understand but I did check AppleLife forum where you posted how to create a codeless kext info.plist for USB ports but it didn't show how to make it for EH01 and EH02 ports.
I can't disable EH01 controller because touchscreen is attached to that port and it's hardwired to it, I can't move it into XHC controller.
The other problem is for desktops (older models).
I have an HP Elite 8300 SFF (Ivy Bridge).
It has 6x USB2 Ports + 3xUSB2 (internal headers) and 4xUSB3 ports.
What to do on this scenario ?
9 USB2 + 4USB3 with 2.0 and 3.0 functionality makes a total of (9+8=17) which is out of the 15 limit.
Should it be possible to implement EH01 and EH02 ports with such kext ?
Do you have an example (template kext) that has EH01 and EH02 ports ?

You didn't read a manual up to the end. There is about Legacy_InternalHub-EHCx.kext
P.S. It not my place, I won't make comments here further.

You didn't read a manual up to the end. There is about Legacy_InternalHub-EHCx.kext
P.S. It not my place, I won't make comments here further.

Thank you @Andrey1970AppleLife , that's all I needed for reference so I can find the issue.
Sorry for bothering you with that, I had to use Google translate on AppleLife to read that.
Will check the entire thread.
Thanks again.

Did you use USBInjectAll.kext when setting up your USB ports? It's a requirement.
Yes, in future please don't bother the acidanthera team with Hackintosh issues.

Did you use USBInjectAll.kext when setting up your USB ports? It's a requirement to show all ports for configuring in Hackintool.
Yes, in future please don't bother the acidanthera team with Hackintool issues.

Yes, I used USBInjectAll.kext + my SSDT-UIAC (SSDT-USB) as well on second attempt so everything is preconfigured and just do the USB ports.kext
The problem is with EH01 and EH02 controller ports.
Specifically EH01 > PR11 part is ok but under PR11 the HP15 port doesn't appear anywhere at all.
Check files on attached USB.zip for reference.
I can't figure out where the problem is.
XHC ports are perfect on every hackintosh.
Just EH01 and EH02 controllers are not being configured properly, tested and verified on 4 hacks so far.

I'm hoping that @corpnewt can take a look at this as I don't have access to any machine with EH0 ports on it.

EHCx Hub ports are from /S*/L*/E*/IOUSBHostFamily.kext/C*/P*/AppleUSBEHCIPCI.kext/C*/Info.plist:
MacBook8,1-EHC1

CFBundleIdentifier
com.apple.driver.AppleUSBMergeNub
IOClass
AppleUSBMergeNub
IONameMatch
EHC1
IOProviderClass
AppleUSBEHCIPCI
IOProviderMergeProperties

kUSBMuxEnabled

model
MacBook8,1

USB devices to plug into the external ports and attach under Hubs can be configured like that in /S*/L*/E*/IOUSBHostFamily.kext/C*/P*/AppleUSBHub.kext/C*/Info.plist:
MacPro6,1-InternalHub

CFBundleIdentifier
com.apple.driver.AppleUSBHostMergeProperties
IOClass
AppleUSBHostMergeProperties
IOProviderClass
AppleUSB20InternalIntelHub
IOProviderMergeProperties

port-count

AQAAAA==

ports

PRT1

port

AQAAAA==

portType
2

board-id
Mac-F60DEB81FF30ACF6
locationID
487587840

These kexts are from 10.14.6.
You can use model instead of board-id, or vice versa.
USB devices listed under Hub ports should have their type set as USB2 (0), not Internal (255) or they will fail to be injected.
My kexts:
USB kexts.zip
My IOReg:
MacBook Pro.zip
I hope you can update Hackintool to check for "hub" ports and "device" ports that are under the "hub" with USBInjectAll.kext installed because currently Hackintool mixes them up in unpredictable ways:
USBPorts.kext.zip
But as Sniki pointed out, it works great for XHC, and also many other things I greatly appreciate :)

EHCx Hub ports are from /S*/L*/E*/IOUSBHostFamily.kext/C*/P*/AppleUSBEHCIPCI.kext/C*/Info.plist:

USB devices to plug into the external ports and attach under Hubs can be configured like that in /S*/L*/E*/IOUSBHostFamily.kext/C*/P*/AppleUSBHub.kext/C*/Info.plist:

These kexts are from 10.14.6.
You can use model instead of board-id, or vice versa.
USB devices listed under Hub ports should have their type set as USB2 (0), not Internal (255) or they will fail to be injected. Oh shoot, I have not tested USB2 (2)...
My kexts:
USB kexts.zip
My IOReg:
MacBook Pro.zip
I hope you can update Hackintool to check for "hub" ports and "device" ports that are under the "hub" with USBInjectAll.kext installed because currently Hackintool mixes them up in unpredictable ways:
USBPorts.kext.zip
But as Sniki pointed out, it works great for XHC, and also many other things I greatly appreciate :)

Thank you for confirming this @whatnameisit ,
As it really makes USB 2 ports useless or better say this awesome kext useless.
With UsbInjectAll.kext in my case PR11 and PR21 are the two USB2 Hubs which should be 255 and the ports are HP11 till HP21 or so.
So what you are saying is that those ports should be as PRT instead of HP right ?
Example as you can see on my files, my Thinkpad X240 does have a EH01 controller as touchscreen on some models is attached to EH01>PR11>HP15 or my model is the smart card reader.
So with the usbports.kext it should be PRT5 ?
Am i correct ?

@Sniki

So what you are saying is that those ports should be as PRT instead of HP right ?

The example screenshots I took from Apple's kexts both have the name PRT1 for both "Hub" and "devices", so it is confusing when you ask if something should be PRT. And port naming does not matter at all, as you can see in my IOReg I have named all "device" ports so I can identify them just by reading their names: H11L for EHC1 left @ 1 and H23W for EHC2 webcam @ 3.

So with the usbports.kext it should be PRT5 ?

Since port names do not matter, I believe it is that your kext needs to set the porttype for the "device" port @ 5 to be "0". And I'm not sure for IOName, CFBundleIdentifier, and IOClass but I modified them to look similar to mine:
USBPorts.kext.zip
I hope it works for you...

@whatnameisit you are a legend, currently im on Arch Linux on my X240 but i did try on my Desktop HP Elite 8300SFF by checking you two separate kexts, you figured it out.
Now it's working perfect, i did manage to make them work in a single kext now including XHC.
Here is the attached ioreg and USBPorts.kext the correct way.
As HUB1 and HUB2 we can actually name them as we wish as you mentioned above.
@headkaze now we have the correct implementation, have a look at this USBPorts.kext to incorporate changes.
Thanks !
HP8300.zip
USBPorts.kext.zip

@whatnameisit can you please contact me (username @ gmail.com) to discuss fixing this issue. First thing that would be useful is to generate a USBPorts.kext using Hackintool then make the minimum changes required for it to work and then show me the changes side by side so I can go through and make the changes. Then I need you to test that Hackintool is generating the file correctly as I don't have a machine with a HUB to test this.

@Sniki @headkaze The way with Hackintool is to install USBInjectAll which then requires EH01, EHC2, and XHC renames, right? If all devices are correctly named and all ports are injected then you have the same situation as having installed USBInjectAll the correct way. Then Hackintool's USB tab works as it should.
Doesn't OpenCore lift the limit on XHCI only and without any renames because disabling EHCI and renames is what's recommended?
Anyways, you guys can collaborate by committing EHC1, EHC2 name match on USBInjectAll, or even better is to match by locationID such as 1D000000 for EHx1 and 1A000000 for EHx2 and the same thing for Intel's and AMD's XHC (It works in either hex or decimal.). Hackintool can check the locationID of these controllers and show all ports on the screen.
I can test :)

@Sniki @headkaze The way with Hackintool is to install USBInjectAll which then requires EH01, EHC2, and XHC renames, right? If all devices are correctly named and all ports are injected then you have the same situation as having installed USBInjectAll the correct way. Then Hackintool's USB tab works as it should.
Doesn't OpenCore lift the limit on XHCI only and without any renames because disabling EHCI and renames is what's recommended?
Anyways, you guys can collaborate by committing EHC1, EHC2 name match on USBInjectAll, or even better is to match by locationID such as 1D000000 for EHx1 and 1A000000 for EHx2 and the same thing for Intel's and AMD's XHC (It works in either hex or decimal.). Hackintool can check the locationID of these controllers and show all ports on the screen.
I can test :)

Good idea, since im maintaining USBInjectAll.kext now, i will try to change so it matches with location id as it's required work to be done only on EHCI controllers.
That way we can avoid renames like EHC1 and EHC2 to EH01 and EH02
Thanks for the idea !

@Sniki I'm wondering if it would be possible to have Hackintool configure ports without the need for USBInjectAll?
Since we know this about the location id:
// The value (e.g. 0x14320000) is represented as follows: 0xAABCDEFG
// AA  - Ctrl number 8 bits (e.g. 0x14, aka XHCI)
// B   - Port number 4 bits (e.g. 0x3, aka SS03)
// C~F - Bus number  4 bits (e.g. 0x2, aka IOUSBHostHIDDevice)
// C~F are filled as many times as many USB Hubs are there on the port.
Does USBInjectAll just rename the ports HSxx, SSxx and HPxx based on location id? If you have a port limit patch without USBInjectAll does IORegistry still list all ports?

@Sniki I'm wondering if it would be possible to have Hackintool configure ports without the need for USBInjectAll?
Since we know this about the location id:
// The value (e.g. 0x14320000) is represented as follows: 0xAABCDEFG
// AA - Ctrl number 8 bits (e.g. 0x14, aka XHCI)
// B - Port number 4 bits (e.g. 0x3, aka SS03)
// C~F - Bus number 4 bits (e.g. 0x2, aka IOUSBHostHIDDevice) // C~F are filled as many times as many USB Hubs are there on the port.
Does USBInjectAll just rename the ports HSxx, SSxx and HPxx based on location id? If you have a port limit patch without USBInjectAll does IORegistry still list all ports?

@headkaze good question.
To be honest im not 100% sure yet.
I will do some testing tonight and tomorrow as it's weekend and i don't have anything else on my agenda.
Thanks for these details
Will check OpenCore XHCI Port Limit removal Quirk without USBInjectAll.kext and see if all ports do get listed, if that turns out to be true, we won't need USBInjectAll.kext anymore.

@Sniki @headkaze USBInjectAll injects all ports with the names defined in its Info.plist by locationID. To be a little bit more informative, macOS first checks the ACPI if any USB ports are active, which many times are wrong. To get around the wrong ACPI, macOS then checks if any ports are injected on the IORegistry level and overwrites the ACPI ports. If no data is injected on the IORegistry level, macOS assumes the ports found in the ACPI.
And here's a bit of what USBInjectAll has in its Info.plist:

I don't think it will be possible to do without USBInjectAll. Because it injects the XHC ports UsbConnector to 3 (USB 3 Standard-A connector) so they'll respond to a device being plugged in. Without it the defaults are set based on (I'm guessing) the Model Identifier. Please correct me if I'm wrong.

I spoke to vit and he seems to think you shouldn't need USBInjectAll with
the XHCIPortLimit Quirk.
Are you using OpenCore + XHCIPortLimit Quirk for your port limit patch?
On Sat, 1 Feb 2020 at 20:52, whatnameisit<EMAIL_ADDRESS>wrote:

@Sniki https://github.com/Sniki @headkaze https://github.com/headkaze
USBInjectAll injects all ports with the names defined in its Info.plist by
locationID. To be a little bit more informative, macOS first checks the
ACPI if any USB ports are active, which many times are wrong. To get around
the wrong ACPI, macOS then checks if any ports are injected on the
IORegistry level and overwrites the ACPI ports. If no data is injected on
the IORegistry level, macOS assumes the ports found in the ACPI.
And here's a bit of what USBInjectAll has in its Info.plist:
[image: 1]
https://user-images.githubusercontent.com/47713349/73592383-1f611880-453d-11ea-8172-ea4eebff44bd.png
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/headkaze/Hackintool/issues/39?email_source=notifications&email_token=AAM6UYNMSVNIATP4LI55QE3RAVWB5A5CNFSM4JHAFO62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKQ4NQA#issuecomment-581027520,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAM6UYNBOZNXXRNVOL7T3RLRAVWB5ANCNFSM4JHAFO6Q
.

I'm using Clover. I'd applied the patch via KextsToPatch. I always had in mind to switch over to OpenCore, so I will later test OpenCore.

Hey @jinmo yang<EMAIL_ADDRESS>if you get some time please
check out Hackintool v3.07.
I've added a routine that will detect and name ports based on locationID so
it may now be able to generate a working kext without the use of
USBInjectAll.
Remember to use the "Clear All" and "Refresh" buttons to get a fresh update
from IORegistry.
On Sun, 2 Feb 2020 at 09:09, whatnameisit<EMAIL_ADDRESS>wrote:

I'm using Clover. I'd applied the patch via KextsToPatch. I always had in
mind to switch over to OpenCore, so I will later test OpenCore.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/headkaze/Hackintool/issues/39?email_source=notifications&email_token=AAM6UYOQEL5B7TQNOXIUKKLRAYMOHA5CNFSM4JHAFO62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKRKZCY#issuecomment-581086347,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAM6UYJUBD3SVBDH5ICZF4LRAYMOHANCNFSM4JHAFO6Q
.

Hey, Ben.
I had all ports already injected via my custom USBPorts.kext. Then Hackintool recognized only the “hub” ports as EHC1-HUB1 and EHC2-HUB1 and exported USBPorts.kext and SSDT-UIAC.aml as if there were only “hub” ports in the IOReg. Also, SSDT-USBX.aml was exported, which I already had installed and booted with.

오후 12:30, headkaze<EMAIL_ADDRESS>작성:

Hey @jinmo yang<EMAIL_ADDRESS>if you get some time please
check out Hackintool v3.07.
I've added a routine that will detect and name ports based on locationID so
it may now be able to generate a working kext without the use of
USBInjectAll.
Remember to use the "Clear All" and "Refresh" buttons to get a fresh update
from IORegistry.
On Sun, 2 Feb 2020 at 09:09, whatnameisit<EMAIL_ADDRESS>wrote:

I'm using Clover. I'd applied the patch via KextsToPatch. I always had in
mind to switch over to OpenCore, so I will later test OpenCore.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/headkaze/Hackintool/issues/39?email_source=notifications&email_token=AAM6UYOQEL5B7TQNOXIUKKLRAYMOHA5CNFSM4JHAFO62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKRKZCY#issuecomment-581086347,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAM6UYJUBD3SVBDH5ICZF4LRAYMOHANCNFSM4JHAFO6Q
.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/headkaze/Hackintool/issues/39?email_source=notifications&email_token=ALMAYROM7ADZNOVFIBYLJ7DRCILVHA5CNFSM4JHAFO62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELLDUIY#issuecomment-584464931, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALMAYRL3ZKHZ4XEC7RDHKW3RCILVHANCNFSM4JHAFO6Q.

Yeah so USBInjectAll is needed to inject the ports but I think we can make a quirk for OC to do the same thing at some point.

@headkaze sorry for my late answer here, i've been busy with other stuff.
I will do some testing soon on the USBPorts side, for now even though it's configured correctly by creating the USBPorts.kext manually, i will give it a try from scratch and see how it works during this week/weekend

Hey, headkaze. Instead of opening a new issue, I came here because of the similarities.
I have an HP Elite X2 1012 G2, and it has an internal hub under XHC that detachable keyboards connect. The keyboard does not show up, and I'm thinking that Hackintool has never supported this kind of setup. A similar setup, however, is used in certain iMac models:
		<key>iMac13,1-InternalHub-EHC1-InternalHub</key>
		<dict>
			<key>CFBundleIdentifier</key>
			<string>com.apple.driver.AppleUSBHostMergeProperties</string>
			<key>IOClass</key>
			<string>AppleUSBHostMergeProperties</string>
			<key>IOProviderClass</key>
			<string>AppleUSB20InternalHub</string>
			<key>IOProviderMergeProperties</key>
			<dict>
				<key>port-count</key>
				<data>
				AQAAAA==
				</data>
				<key>ports</key>
				<dict>
					<key>PRT1</key>
					<dict>
						<key>port</key>
						<data>
						AQAAAA==
						</data>
						<key>portType</key>
						<integer>2</integer>
					</dict>
				</dict>
			</dict>
			<key>locationID</key>
			<integer>488112128</integer>
			<key>model</key>
			<string>iMac13,1</string>
		</dict>

This specific code above means there is an internal hub (locaionID of 488112128 or 0x1D180000) alongside other ports under EHCI (the controller), and a lower port is connected to this internal hub. This should apply to boards that have internal hubs.
Since the same driver AppleUSB20InternalHub is shown in IORegistry with no injector kext, I thought I might use this structure for my XHC's internal hub, and it worked:
		<key>MacBookPro14,1-XHC-internal-hub</key>
		<dict>
			<key>CFBundleIdentifier</key>
			<string>com.apple.driver.AppleUSBHostMergeProperties</string>
			<key>IOClass</key>
			<string>AppleUSBHostMergeProperties</string>
			<key>IOProbeScore</key>
			<integer>5000</integer>
			<key>IOProviderClass</key>
			<string>AppleUSB20InternalHub</string>
			<key>IOProviderMergeProperties</key>
			<dict>
				<key>port-count</key>
				<data>AwAAAA==</data>
				<key>ports</key>
				<dict>
					<key>HS01</key>
					<dict>
						<key>name</key>
						<string>ABCD</string>
						<key>port</key>
						<data>AQAAAA==</data>
						<key>portType</key>
						<integer>0</integer>
					</dict>
				</dict>
			</dict>
			<key>locationID</key>
			<integer>337641472</integer>
			<key>model</key>
			<string>MacBookPro14,1</string>
		</dict>

I'm attaching IOReg below.
Jinmo’s MacBook Pro.zip
I thought making searching for internal hub could be similar to searching for ports under EHCI. Would you like to update the USB tab like last time?


Iox #78 Minor RouDi config adaptions

Make RouDi_GTest class more configurable
Reflect RouDiConfig changes in TOML files

Fine so far, but here, the old default config is hardcoded
https://github.com/eclipse/iceoryx/blob/master/doc/usage-guide.md
Can you replace this with a link instead of having a config written there?

Oh, and we have to adapt the ipceperf example to not use the large chunks (or ensure a proper mempool config)


add MIR changes to Alonzo spec

Additionally, added a new check to the Alonzo MIR rule that ensures that the Coin valued used in transfers (reserves <-> treasury) is always non-negative. This does mean that the predicate serialization has changed, but as this is a governance feature we should be fine (cc @nc6).

I somehow broke the latex build, I will fix that tomorrow


Imported 0 of out 0 events

Followed all the steps, but it didn't extract my schedule from AMIS. Is it because I'm a freshman?

What browser are you using? Also have you waited till the marked "STATUS" is "READY" before clicking download?

I'm using google chrome, and yes I did wait for the "READY" status. Some freshmen were also having the same issue, so I hypothesized that maybe there was a correlation.

I see. Thank you for raising the issue. I will be looking into this. I did have successful testing, however none of those incldued freshmen.
What is the status of the subjects you are given? It may have something to do that you freshmen don't really enlist on your 1st semester here in the university. The program essentially checks first for any subjects with the "Enlisted" property like the image below:

Good day. I have found from another source that for freshman, the course status for all your courses is "Finalized", not "Enlisted". I will roll out a changed later.
If you can, kindly notify your batchmates that if they still wish to use the extension, there would be a fix available later. Thank you!


Load balancing and Port forwarding are not displayed properly

Fixes #766
For isolated networks, load balancing, port forwarding and firewall
options are not displayed.
For source nat ip of vpc, all options are displayed but it should not be
Below are the expected actions according to legacy UI
for isolated networks
if static nat is disabled then display all options
if static nat is enabled then display only firewall tab
for VPC networks
for source nat IP, dont display any option
for other ip's firewall is always disabled.
Load balancing and port forwarding options are mutually exclusive

@blueorangutan package

@davidjumani a Jenkins job has been kicked to build primate packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark:centos :heavy_check_mark:debian :heavy_check_mark:archive.
QA: http://primate-qa.cloudstack.cloud:8080/client/pr/768 (JID-3546)


User/Instance Growth Statistics

I have this data available from the Lemmy Explorer dumps, which can show instance growth numbers for each instance.
This ticket would integrate this data into Lemmy Modder, to allow mods to:

View User Growth
View Post/Comment Growth

More difficult, and maybe related to #50 - we could pull additional statistics directly from the Lemmy DB, for example:

Number of reports dealt with in the past ~week
Number of reports resolved without content removal
Most active mods (maybe also data from modlog)

From https://beehaw.org/post/574562

I have some thoughts about the risk and possible performance issue if you would directly hook into the database of an lemmy instance. Could be a security risk if wrongly implemented. Or just a performance bottleneck.
I would see a possibility in #60  to make this possible. For example that it could use its own source, like a pre cached database, static file that gets updated or just direct access. They can choose directly who has access to that data.
And the responsibility goes further away from you of dealing with spam, authorisation and so on.

This feature would be sources solely from the lemmyverse data exports.
This would be a new "home" page that all users have access to. This would just report on the latest instance metrics and stats/versions, etc.


added support for u2f migrated appId

This PR solves issue https://github.com/webauthn-open-source/fido2-lib/issues/62
Example usage:
Fido2Lib.addExtension('appid', (extName, type, value) => value, () => { }, () => { })

const f2l = new Fido2Lib({ ... })
f2l.enableExtension('appid')

// challenge
const authOpts = await f2l.assertionOptions({
  extensionOptions: {
    appid: 'https://www.example.com'
  }
})
authOpts.challenge = coerceToBase64Url(authOpts.challenge, 'challenge')
const authChallenge = {
  allowCredentials: [
    {
      id: 'Vqc7...',
      type: "public-key"
    }
  ],
  ...authOpts
}
console.log(JSON.stringify(authChallenge, null, 2))
/* output
{
  "allowCredentials": [
    {
      "id": "Vqc7...",
      "type": "public-key"
    }
  ],
  "challenge": "hF6cDwH8Xy6uAkp2ivBIL9Fla4_HNptF5nbtxfHpz2sajVvU4GIHUzY43zIGnU2AGARQp0tD-aiaU3Zecw4ocA",
  "timeout": 60000,
  "rpId": "example.com",
  "userVerification": "preferred",
  "extensions": {
    "appid": "https://www.example.com"
  }
}
*/

// verification
const expectedAuth = {
  challenge: authChallenge.challenge,
  origin: "https://www.example.com",
  rpId: authChallenge.extensions.appid,
  factor: "either",
  publicKey: jwkToPem(coseToJwk(coerceToArrayBuffer('pQECAy...', 'pkey'))),
  prevCounter: 1234,
  userHandle: null
}

authResponse.id = coerceToArrayBuffer(u2fResponse.id, 'id')
const authResponse = {
  "id": "Vqc7...",
  "response": {
    "authenticatorData": "lK8QzA8NSNiTPCRHJiqlEt4CmIbp1e_wbVZ04wfB8I0BAAAFlw",
    "clientDataJSON": "eyJ0eXBlIjoid2ViYXV0aG4uZ2V0IiwiY2hhbGxlbmdlIjoiaEY2Y0R3SDhYeTZ1QWtwMml2QklMOUZsYTRfSE5wdEY1bmJ0eGZIcHoyc2FqVnZVNEdJSFV6WTQzeklHblUyQUdBUlFwMHRELWFpYVUzWmVjdzRvY0EiLCJvcmlnaW4iOiJodHRwczovL3d3dy5zdGFnaW5nLmJpdGZpbmV4LmNvbSIsImNyb3NzT3JpZ2luIjpmYWxzZX0",
    "signature": "MEQCIE_-3E2ZWKs7hdzoV4MqZhbdto4ipqiyHok5fYjxNTE6AiBI4rD6kV_nb4ETc935yCjY1LFCcphnEZtf6FnF3n_YMw"
  }
}

const authRes = await f2l.assertionResult(authResponse, expectedAuth)

const valid = authRes.audit.complete && authRes.audit.validRequest && authRes.audit.validExpectations
const newCounter = authRes.authnrData.get('counter')
console.log(valid && newCounter > dbRecord.counter, newCounter) // true, 1235

Hi @JamesCullum , thanks again for this very useful package. When possible could you take a look at this PR and issue?
I was able to produce this one locally and it worked with u2f migrated api credentials that I've got before

Thanks for the added input.
I would prefer if we would do a single validation of the URL, and inside that we can do a check of both a URL and a domain.
However I'm not a big fan of validating a URL by simply passing it to a URL object, as such a constructor normally tries to correct input, while we want to have a more strict validation.
Also as part of this change I think we should add the example that you posted into the documentation.

Thanks for the added input.
I would prefer if we would do a single validation of the URL in a joined method, and inside that we can do a check of both a URL and a domain.
However I'm not a big fan of validating a URL by simply passing it to a URL object, as such a constructor normally tries to correct input, while we want to have a more strict validation.
Also as part of this change I think we should add the example that you posted into the documentation.

Agree, we can create a simple util for validating full domain and url, e.g. isValidDomainOrUrl. Alright I'll try to use more strict validation for url by adding some rules e.g. https, allow path etc.
Sure I'll update the docs with this example, should I add it on readme or somewhere else?

Maybe we can have a full sample file somewhere and link to it in the readme?

Maybe we can have a full sample file somewhere and link to it in the readme?

oh I've added it to readme, but if you want I can move it to a file like examples/u2f-migrated-example.js?

Yes please, the example is a bit too long for a README I'd say.

Yes please, the example is a bit too long for a README I'd say.

Hey @JamesCullum, just moved the example from readme to js file

Awesome changes, thanks a lot - looks really good and clean now!
As an extra feature it would be great if we could use the new dynamic function to allow the user to config (opt-in of course) the rules - but thats a different feature for the future! If you're on a run you can check if you think this makes sense, otherwise already thanks a lot!

Awesome changes, thanks a lot - looks really good and clean now!
As an extra feature it would be great if we could use the new dynamic function to allow the user to config (opt-in of course) the rules - but thats a different feature for the future! If you're on a run you can check if you think this makes sense, otherwise already thanks a lot!
EDIT: Travis is back and found some issues - would be great if you could resolve them

Yeah we can do that in separate PR and adjust the validators to be more flexible based on user needs. Also just fixed lint issue, not sure how I missed it :sweat_smile:. Could you approve workflow tasks please so we can check if it's all good for merge and release?

Codecov Report

Merging #63 (db7e0d6) into master (e03c2df) will increase coverage by 0.18%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master      #63      +/-   ##
==========================================
+ Coverage   87.07%   87.25%   +0.18%
==========================================
  Files          14       14
  Lines        1826     1852      +26
==========================================
+ Hits         1590     1616      +26
  Misses        236      236

Impacted Files
Coverage Δ

lib/utils.js
84.39% <100.00%> (+3.52%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e03c2df...db7e0d6. Read the comment docs.

thanks a lot @JamesCullum for the merge, could we make also a npm release please?

Sure - next time you can as well directly bump the version, so that it creates an npm release on merge 👍


[BUG] No more output on shared chatbot v2.1.0

Describe the bug
Simple bug; None of my shared chatbot are able to output the answer
The query is fetch, I have a working state in langsmith, so the flow is working; but in the share chatbot window, the output is not visible.
I ask my query, i see the tool "thinking", and it stop with no output print on screen.
This is only since this morning update to 2.1.0

BUG solved with 2.1.1 Thanks very much


SECURITY VULNERABILITIES - Regular Expression Denial of Service

Dependencies report Regular Expression Denial of Service in momentjs library
see here - https://david-dm.org/uqlibrary/fez-frontend

Duplicate of https://github.com/moment/moment/issues/4163.
Fix at https://github.com/moment/moment/pull/4326.


HealthList wont come up

Hey not sure if others are having this issue with the health list HUD's via tibia11 client. But
I just re downloaded tibialyzer and got every set up as I wanted again, but then I went to set my HUD's for my health list, being a ED this is super handy and for some odd reason whether its my simple error or the program its self, but I feel I'm doing everything to a T, from yours (Mytherin) and (Erics) how to videos
if there's anyone with a solution or experiencing the same issue would really appreciate the help or some advice. All in all this is a really great program and love your work, if there's a way I can donate to you (Mytherin) to keep up with the development, feel free to link me it here
thanks and hope for a speedy answer
yours truly
Dro

I have the same problem.
I place the name on the HealthList, click on "show hud" but nothing happens.


[WIP] Use ThreadLocal to store a database transaction handle

Moved this pull request to https://github.com/treasure-data/digdag/pull/515


[OTHER] Digital Accessibility Toolkit - Links in Contribute on Github.com are opening in the same tab

Describe the purpose of this issue
On the Digital Accessibility Toolkit site in the "Contribute on Github.com" banner, I noticed that the Github links redirects the current page rather than opening in a new tab.
I believe it is best practice to have external links open in a new tab. This practice will also allow the user to navigate the page that they intend on flagging an issue on without the need of opening a new tab and finding it again.
Thank you!

Thank you for sharing your source! I am embarrassed to say that I have been applying the wrong practices.


Add link to FreeNAS tutorial

Forgot to add a link in README.md for the FreeNAS doco. Also renamed the file to match the docker tutorial.

thanks :)


Region selection

@rauchg indicated that I can "handle the region affinity, you'll just have to add a field to package.json under the now namespace." What field should I use to do this?
@aexmachina

No features in here

Hey there! I am facing the same situation. @leo closed his issue, but what is the outcome?
Can I set the region somehow?
Thx! :)

I was told by @rauchg:

To handle the region affinity, you'll just have to add a field to package.json under the now namespace.

But I never managed to find out the name of the field 😉

We'll be documenting this very soon! Since this is a feature we're working on secretly, this is the only thing I can tell you... 😊

+1 region selection

We're on it!

Are there any updates to this? Region latency from Australia/Sydney pretty bad right now and would love to be able to change this to default to region closer to where my customers are.

I'm having critical latency issues too. Could this be related to a region ? #592

+1

+1 for region selection from Sydney.

+1 for Singapore come on @rauchg you were here just last year lol

+1 us-east-1

+1 For Europe

+1 for Tokyo

Is there any estimate for when the Brussels region will be available? I'm trying to persuade my client to use Now but they are reluctant without a European region to deploy to.

We have a region in EU (in Brussels) that you can deploy to. Check out https://bru.now.sh/ to test your latency to BRU.
To deploy to BRU, you should be able to run:
now --regions bru

OK great - thanks!

Just to be clear, BRU is not in general availability yet. When it's available you'll be able to do something similar to what @pranaygp's showing 👍

Now.sh introduced the BRU1 (Belgium) region 2 days ago. https://zeit.co/blog/multi-dc


ENABLE_BITCODE error

ld: '/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/Library/Frameworks/XCTest.framework/XCTest' does not contain bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target. for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
I tried "ENABLE_BITCODE = No" it dint work, any help would be great!

Hi @BMe2U,
We've never seen this issue before. It doesn't look related to this library however. Did you manage to find a fix for it?

I'm closing this. Please let us know if you still have an issue while trying to use this library.

Faced the same issue. It happens when you select a device instead of simulator and test. Even the sample project also fails to build for device.

@ilife I've made a PR to disable bitcode (https://github.com/net-a-porter-mobile/XCTest-Gherkin/pull/39). This will allow you to run on hardware devices.

I can see the ENABLE_BITCODE as "Yes", so i'm still getting the error.

That's very strange :/ Quick sanity check - you've re-run pod install?

A new version hasn't been released with #39 yet. @nap-sam-dean can you cut a new release from master? and @BMe2U can you try to point the podfile to HEAD and check if this solved your problem?

My fix for this was to disable bitcode on the targets specified in project build settings.

This has been pushed out as version 0.7.0 - let me know if it solves the problem.
@nmcdonaldd Do you have an issue with having to reset that property each time you do a pod install / pod update?

@BMe2U Hi - it's been 21 days sine any chat on this issue, we think it's fixed so unless you can confirm that it's not I'm going to close this issue in the next few days, thanks!

I'm closing this because we think it's fixed and the OP has gone quiet on the matter.


Instructions on how to run

Hi!
I hope you are doing well.
I tried running the code in the examples directory, this is the result:

$ cargo build
 Downloading winapi v0.1.17
 Downloading log v0.3.1
 Downloading libc v0.1.6
 Downloading user32-sys v0.1.0
 Downloading kernel32-sys v0.1.0
 Downloading gdi32-sys v0.1.0
   Compiling gdi32-sys v0.1.0
   Compiling user32-sys v0.1.0
   Compiling kernel32-sys v0.1.0
   Compiling libc v0.1.6
   Compiling examples v0.0.1 (file:///E:/files/dev/rust/tests/icon/rust-windows-master/examples)
   Compiling log v0.3.1
   Compiling winapi v0.1.17
   Compiling rust-windows v0.0.1 (file:///E:/files/dev/rust/tests/icon/rust-windows-master)
src/hello.rs:23:5: 23:13 warning: unused import, #[warn(unused_imports)] on by default
src/hello.rs:23 use std::env;
                    ^~~~~~~~
src/hello.rs:198:9: 198:18 warning: unused variable: `exit_code`, #[warn(unused_variables)] on by default
src/hello.rs:198     let exit_code = main_window_loop();
                         ^~~~~~~~~

Can you help me out?

If I run from the main directory, cargo build works but cargo run says:
$ cargo run error: a bin target must be available for cargo run

I made a mistake in running...


The content of the keyboard shortcuts table is not painted correctly

Problem Description
When the size of section Name is reduced, the text is painted beyond the size of the section.

Versions

Spyder 4.0.0.dev0 (Commit: 51a29bb3a)
Python 3.7.3 64-bit | Qt 5.9.6 | PyQt5 5.9.2 | Windows 10

So this problem arises because we paint the content of section Name with HTMLDelegate
A solution would be to set correctly the rectangle on which the text is painted in the HTMLDelegate paint method with:
painter.setClipRect(textRect.adjusted(0, 0, -self._margin, 0))
However, this doesn't look perfect because the text is not elided like it is for the other two columns. I think eliding the text wouldn't be easy because it is rich text and this functionality is not supported by QTextDocument.
https://bugreports.qt.io/browse/QTBUG-34857

Personnally, I would fix this by setting the resize mode on sections Context and Name so that their width is fixed to fit their content. IMO, there is not a really big plus value in allowing the user to resize the width of the Context and Name columns here.

Personnally, I would fix this by setting the resize mode on sections Context and Name so that their width is fixed to fit their content. IMO, there is not a really big plus value in allowing the user to resize the width of the Context and Name columns here.

Ok, I agree with this. Thanks for taking a look at it this @jnsebgosselin!

@jnsebgosselin also in dark mode we are not changing the color of the text, I think we could also fix the HTML delegate to take this into account


1037 - Cache clear mechanism

This adds a DELETE /cache/{cache name}/clear endpoint that can only be accessed locally which clears the named cache.

 Add mechanism for clearing API caches without releasing a new build


Retaining old messages

Is it possible to publish the messages in "retain" mode (where, the subscribers who join late do not miss the earlier posted messages and get them immediately upon subscription).
For example, publisher created with "retain" mode (and optional buffer length, say n), keeps track of the last n posted messages and pushes them immediately to all new subscribers as and when they signup.
It would be very useful function to support, as it helps in syncing the producer-consumer properly.
MQTT is good example of where this is supported, and is very useful, especially in bridged/forwarded cluster scenarios.

Crossbar.io and WAMP currently do not support any kind of message persistence. (There's a WAMP issue for this - https://github.com/wamp-proto/wamp-proto/issues/43).
Currently the way to go here is to have a component which subscribes to the messages, persists them, and which offers a procedure for newly connecting clients to request back messages (e.g. based on a timestamp).
Closing this issue here since it does not concern the crossbar examples.


⚠ Bofrost.lu has degraded performance

In bf9797b, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 2725 ms

Resolved: Bofrost.lu performance has improved in e7978f6 after 6 minutes.


Inline environment variables are treated as a command

Summary
Running this program with a command that accepts environment variables will fail if these environment variables are provided inline before the command.
For example, NUMBER=4 is treated as the command in the following:
$ ./etime -p NUMBER=4 ./square

Which outputs the following (omitting the correct measurements):
time: NUMBER=4: No such file or directory

Ideally this would be set as an environment variable for the example square command!
Possible workarounds
Define the environment variables before etime:
$ NUMBER=4 ./etime -p ./square

Or temporarily set environment variables with export and unset:
$ export NUMBER=4
$ ./etime -p ./square
$ unset NUMBER

Fixed in #78 by running commands in a subshell! Both of the workarounds are still valid and now the example command works as expected.


Generate Parentheses in Java

💥 Proposal
I would like to add a program that will generate combinations of n pairs of parentheses.
Have you read the Contributing Guidelines on Pull Requests?
Yes.

/assign


Support building and running unit tests with CMake toolchain

Context:

CMake all target was actually not building tests with -DBUILD_TESTS=ON. Tests were implemented only in make toolchain. Some IDE and developers could not easily work with current cmake/ctest toolchain.

Changes:

new build_tests and check cmake targets which build and run all unit tests (61 as of now). Support of both options BUILD_TRAINING_TOOLS and DISABLED_LEGACY_ENGINE
BUILD_TESTS option has been replaced by the standard CTest BUILD_TESTING
CMake Github workflow has been updated to reflect these changes, and a new check unittest step is added on linux os only
unittest/README.md has been updated to document cmake toolchain check target

Limitations:

[ ] Github workflow cmake-win64.yml is not updated because it is required to install (at least?) icu before test and training compilation.
[ ] Github workflow cmake.yml does not execute unittest for MacOs because some fonts are missing: FontUtilsTest.* are failing. One can probably easy fix it later on.
[ ] TENSORFLOW option yet to be supported in cmake toolchain.
[X] It is required to enable options -DBUILD_TRAINING_TOOLS=ON and -DBUILD_TESTING=ON altogether as in the make toolchain, tests cannot compile without it.

Checks:

[X] GitHub workflows cmake.yml and cmake-64.yml have been run without  regression.

IMO cmake unit tests should be splited into parts based on required dependencies. e.g. ICU for Windows/VS studio could be downloaded automatically, but Pango for VS code not (+you can not build it with cmake unless you).
On the other hand, mingw/msys2 should have no problem...
Also, it makes sense for me if I can skip Tesseract library tests if I do modifications only in training tools.
I doubt if it makes sense to run cmake unitest as github actions - there are already running unit tests, that will catch potential problems anyway.

Closing as it requires additional effort mentionned by @zdenop and cannot be merged as is.


App Entity not showing model driven apps

The app entity view "All Apps" does not show all apps only canvas. The filters are Active and not deleted, somewhere the model driven apps must not have this field or its set wrong.
Version used was downloaded today and should be 1/0/0/0

There was an issue with marking Model Driven Apps as deleted. I have fixed this now in the development environment, and it will be published with the next new version (likely next week).

Fixed


Fix numpy install (3.12)

This should work to get numpy build with Python 3.12.
A bit more involved than usual, but until https://github.com/scikit-build/ninja-python-distributions/issues/220 is resolved, ninja needs to be build from git directly unfortunately.
/CC @frenck

Example run from my fork: https://github.com/cdce8p/wheels/actions/runs/6444260435
Just not sure how to fix the lint issue:
Error: Dockerfile:17 DL4006 warning: Set the SHELL option -o pipefail
  before RUN with a pipe in it. If you are using /bin/sh in an alpine image
  or if your shell is symlinked to busybox then consider explicitly setting
  your SHELL to /bin/ash, or disable this check

Pushed up change, that addressed the Hadolint issue. Could you check if it still works as expected?

Pushed up change, that addressed the Hadolint issue. Could you check if it still works as expected?

Thanks! Looks good AFAICT. The workflow is still running but the first jobs were successful already. I'd say, let's try it.
https://github.com/cdce8p/wheels/actions/runs/6447285222/job/17503378562


A terminal window popup showing on opening app - Windows

On windows release app it showing a terminal window popup and closing automatically.

To avoid terminal windows showing up, I change void PlatformDeviceIdWindowsPlugin::HandleMethodCall in platform_device_id/platform_device_id_windows/windows/platform_device_id_windows_plugin.cpp to following:
void PlatformDeviceIdWindowsPlugin::HandleMethodCall(
    const flutter::MethodCall<flutter::EncodableValue> &method_call,
    std::unique_ptr<flutter::MethodResult<flutter::EncodableValue>> result) {
  std::string strResult;
  std::string deviceId;

  if (method_call.method_name().compare("getDeviceId") == 0) {
    std::string cmd = "wmic csproduct get UUID";
    HANDLE hPipeRead, hPipeWrite;
    SECURITY_ATTRIBUTES saAttr = {sizeof(SECURITY_ATTRIBUTES)};
    saAttr.bInheritHandle = TRUE; // Pipe handles are inherited by child process.
    saAttr.lpSecurityDescriptor = NULL;
    CreatePipe(&hPipeRead, &hPipeWrite, &saAttr, 0);
    STARTUPINFOW si = {sizeof(STARTUPINFOW)};
    si.dwFlags     = STARTF_USESHOWWINDOW | STARTF_USESTDHANDLES;
    si.hStdOutput  = hPipeWrite;
    si.hStdError   = hPipeWrite;
    si.wShowWindow = SW_HIDE; // Prevents cmd window from flashing.
    PROCESS_INFORMATION pi = { 0 };
    BOOL fSuccess = CreateProcessW(NULL, (LPWSTR)towstring(cmd).c_str(), NULL, NULL, TRUE, CREATE_NEW_CONSOLE, NULL, NULL, &si, &pi);
    if (! fSuccess) {
        CloseHandle(hPipeWrite);
        CloseHandle(hPipeRead);
    } else {
      bool bProcessEnded = false;
      for (; !bProcessEnded ;) {
          // Give some timeslice (50 ms), so we won't waste 100% CPU.
          bProcessEnded = WaitForSingleObject( pi.hProcess, 50) == WAIT_OBJECT_0;
          // Even if process exited - we continue reading, if
          // there is some data available over pipe.
          for (;;)
          {
              char buf[1024];
              DWORD dwRead = 0;
              DWORD dwAvail = 0;
              if (!::PeekNamedPipe(hPipeRead, NULL, 0, NULL, &dwAvail, NULL) || !dwAvail || !::ReadFile(hPipeRead, buf, min(sizeof(buf) - 1, dwAvail), &dwRead, NULL) || !dwRead)
                  break;
              buf[dwRead] = 0;
              strResult += buf;
          }
      }
      CloseHandle(hPipeWrite);
      CloseHandle(hPipeRead);
      CloseHandle(pi.hProcess);
      CloseHandle(pi.hThread);
    }
    std::size_t pos = strResult.find("\n");
    deviceId = strResult.substr(pos+1);
    result->Success(flutter::EncodableValue(deviceId));
  } else {
    result->NotImplemented();
  }
}

This solution works for me.

build error C2660: 'CreateProcessW': function does not take 9 arguments [D:\Project\App\Flutter\app-desktop\build\windows\plugins\platform_device_id_windows\platform_device_id_windows_plugin.vcxproj]

I also encountered this problem, I fixed it in the following way, parsing the command directly at the dart layer.
    final output = await Process.run(
      'wmic',
      ['csproduct', 'get', 'UUID'],
    );
    var uuid = output.stdout
        .toString()
        .replaceAll(RegExp('\r|\n|\\s|UUID'), '')
        .trim();
    debugPrint('uuid: $uuid');

Thanks, very helpful. However, the latest version has not been updated to pub.dev, so I can only change it myself.


Pool the overflow list in sparseStore.trimLeft

What does this PR do?
Reduce allocations in sparseStore.trimLeft by pooling the overflow list. This code was identified as a hotspot as part of some agent performance work that I'm doing. (There's a writeup in a notebook here.)
Motivation
This improves agent's performance in the uds_dogstatsd_to_api SMP experiment by around 40%.

@songy23 Can you offer advice on the CI failure here? https://github.com/DataDog/opentelemetry-mapping-go/actions/runs/5979204656/job/16222818554?pr=157

No idea but it doesn't seem relevant. This PR doesn't introduce breaking changes AFAICT.


Repackaged classes (grpc for example) still imports the original classes

It's easier to show an example.
Here is the beginning of the re-packaged Logproto class:
package com.github.loki4j.pkg.loki.protobuf;

import com.github.loki4j.pkg.google.protobuf.GoGoProtos;
import com.github.loki4j.pkg.google.protobuf.Timestamp;
import com.github.loki4j.pkg.google.protobuf.TimestampOrBuilder;
import com.github.loki4j.pkg.google.protobuf.TimestampProto;
import com.google.protobuf.AbstractParser;
import com.google.protobuf.ByteString;
import com.google.protobuf.CodedInputStream;
import com.google.protobuf.CodedOutputStream;

The problem here is that these classes still reference to the original GRPC classes (which might or might not present on the classpath).
In our case we have these classes, but we still use GRPC 3.17.3 while loki-logback-appender uses 3.18.1, so we have exceptions due to missing methods.
Here is an example:
Exception in thread "loki4j-encoder-0" java.lang.NoSuchMethodError: 'boolean com.google.protobuf.GeneratedMessageV3.isStringEmpty(java.lang.Object)'
	at com.github.loki4j.pkg.loki.protobuf.Logproto$StreamAdapter.getSerializedSize(Logproto.java:7435)
	at com.google.protobuf@3.17.3/com.google.protobuf.CodedOutputStream$UnsafeDirectNioEncoder.writeMessageNoTag(CodedOutputStream.java:2001)
	at com.google.protobuf@3.17.3/com.google.protobuf.CodedOutputStream$UnsafeDirectNioEncoder.writeMessage(CodedOutputStream.java:1974)
	at com.github.loki4j.pkg.loki.protobuf.Logproto$PushRequest.writeTo(Logproto.java:299)
	at com.github.loki4j.client.writer.ProtobufWriter.endStreams(ProtobufWriter.java:85)
	at com.github.loki4j.client.writer.ProtobufWriter.serializeBatch(ProtobufWriter.java:45)
	at com.github.loki4j.client.pipeline.DefaultPipeline.writeBatch(DefaultPipeline.java:227)
	at com.github.loki4j.client.pipeline.DefaultPipeline.encodeStep(DefaultPipeline.java:208)
	at com.github.loki4j.client.pipeline.DefaultPipeline.runEncodeLoop(DefaultPipeline.java:173)
	at com.github.loki4j.client.pipeline.DefaultPipeline.lambda$start$3(DefaultPipeline.java:111)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

Thank you for your work!

Hi @zalavaari! Thanks for you report again!
Yeah, given that protobuf versions 3.x are incompatible with each other, it's really hard to use loki4j in protobuf mode if there are other protobuf dependencies. I wonder how other java libs that provide protobuf-generated classes deal with this.
The obvious solution is to repackage entire protobuf runtime under com.github.loki4j.pkg. But I would like to avoid this as protobuf support is optional for loki4j.
Another solution is to extract protobuf-generated classes into a separate jar and provide several versions of this jar compiled for the different versions of protobuf (e.g. 3.15.x, 3.16.x, etc.) And even if there would be no pre-compiled jar for a particular version of protobuf, these who really need it could generate Logproto classes on their own and include them into the classpath instead of this jar without any conflicts. This also won't affect these who use loki4j in json mode. Currently I tend to explore this alternative.
If anyone else has any other suggestions feel free to put them in comments for this issue.

Hm. I like the idea of the separated grpc jars. I guess the result would be also a smaller, grpc-free loki-logback-appender base jar for those who using json only. The question is the maintenance cost of the added complexity.
I don't have Gradle experiences, but based on Maven I guess it would be feasible.


Identify List of 5-10 potential blogs posts

A value level validation use case, where admiral is used to validate a small chunk of the data:

validate a parameter by using admiral::derive_param_computed.
Derivation text: derive new parameter/records for ADEX.PARAMCD = ADPCYMG (Actual Dose per Cycle) based on existing parameters - Actual Dose per Cycle (ADPCYMG) = Total Amount of Dose (TOTDOSE) / Number of Cycles (NUMCYC);
admiral::derive_param_computed can add a parameter computed from the analysis value of other parameters.
In this example,  ADEX.AVAL (where ADEX.PARAPMCD = ADPCYMG)
= ADEX.AVAL (where ADEX.PARAPMCD = TOTDOSE) / ADEX.AVAL (where ADEX.PARAPMCD = NUMCYC)

# read in data
path_ads <- "xxxx"
adex <- haven::read_sas(paste0(path_ads, "adex.sas7bdat")) %>%
  select(USUBJID, PARAM, PARAMCD, AVAL)

# subset input data to PARAMCD = "TOTDOSE", "NUMCYC"
adex_TOTDOSE_NUMCYC <- adex %>% filter(PARAMCD %in% c("TOTDOSE", "NUMCYC"))
# admiral::derive_param_computed()
val <-
  derive_param_computed(
  adex_TOTDOSE_NUMCYC,
  by_vars = exprs(USUBJID),
  parameters = c("TOTDOSE", "NUMCYC"),
  analysis_value = AVAL.TOTDOSE / AVAL.NUMCYC,
  set_values_to = exprs(
    PARAMCD = "ADPCYMG",
    PARAM = "Actual Dose per Cycle (mg)"
  )
)

# compare with primary
diffdf::diffdf(adex %>% filter(PARAMCD == "ADPCYMG"),
               val %>% filter(PARAMCD == "ADPCYMG"),
               keys = c("USUBJID", "PARAMCD"))

Love this already - we got a our first post!  I was thinking that maybe we should to 4 posts to fill out the site and keep it form looking sad - then switch to 1 every two weeks.  @pharmaverse/commswg


[Improvement][Module Name] Subprocess Gantt chart

Search before asking

[X] I had searched in the issues and found no similar feature requirement.

Description
Hello~ When we use sub-process components, we can only see the overall running time of this sub-process, but cannot intuitively get the execution time of each step in the sub-process. Could we add a Gantt chart so that it can also display the execution time of each task inside the sub-process? Thanks!
Are you willing to submit a PR?

[x] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

I'm willing to join! But, it might be difficult for me :( Could you give me some ideas, like where should I start ? Thanks!


Won't delete ":" characters

Hi, thank you so far for your perfect tool. Unfortunately it not removes some ":" signs.
As example if you remove SDH on this sub here
https://www.opensubtitles.org/en/subtitles/8617649/s-w-a-t-sins-of-the-fathers-en
You will be left with ":" signs see picture:
and it not removes the credits and the upperscale word "TOYOTA"
Please fix if possible.

Hey there,
thank you for the kind words :)
I have fixed the ':' issue, as well as the 'Captioning sponsored by' and 'Captioned by' subtitles.
Unfortunately as subtitle-filter parses each subtitle entry separately, I don't think it would be possible to create a rule to remove the 'and TOYOTA' line without being too restrictive on normal subtitle text.
New release version is v1.4.1 you should be able to just pip install --upgrade subtitle-filter.


#swagger.ignore break in 2.21.1

We use swagger.ignore for some endpoints, and for some reason it doesn't work anymore.
exports.quotation = async (req, res, next) => {   // #swagger.ignore = true };
I downgrade to 2.21.0 to fix this.

Hi @balouk. Thank you for opening the issue.
Could you send a sample project that reproduces this problem? That way is easier to identify and fix the problem.

Hi @balouk! Sorry for the delay.
Thank you for sending the sample code. I'll take a look and try to fix it as soon as possible.

Hi ! Same thing on 2.21.0 and 2.21.5, // #swagger.ignore = true doesn't work for me.

Hi there!
The version 2.21.6 has fixed this. But if the problem persists, let me know.


Specify and allow users to specify TeX "page width"

Otherwise LaTeX inserts never seem to wrap

as a first step, just enforce a page width of like 25% width of the page. specifying can come much much later

Your suggestion seems harder? I want to send a hard-coded value to mathjax, however that happens. And then use a localStorage value instead if one is set


feat: issuer onboarding for OCI keys

@waltkb im getting this error : Failed after 4 retries: Invalid symbol ' '(40) at index 0


Add warning if potentially-scrapable examples are skipped due to dev-dependencies

What does this PR try to resolve?
Another point of feedback I've received on the scrape-examples feature is that the dev-dependency situation is quite confusing and subtle. To make users more aware of the issue, I added a warning where Cargo will alert users when examples are skipped due to a dev-dependency requirement, along with proposing a fix.
How should we test and review this PR?
The test docscrape::no_scrape_with_dev_deps has been updated to reflect this new warning.
r? @weihanglo
(PS thank you for the reviews Weihang. I know I'm doing lots of little patches right now to get this feature finalized. If you want to share the reviewing burden on scrape-examples with anyone else, let me know!)

Sounds reasonable but not sure if it will become too verbose. Could we have a link to the original feedback as a reference?

Sure, for instance here's a user whose examples weren't being scraped until they added this configuration: https://www.reddit.com/r/rust/comments/zosle6/feedback_requested_rustdocs_scraped_examples/j0qnmu5/
Right now users would only encounter this message if they enable scraping with -Zrustdoc-scrape-examples. Once this feature becomes more well-known and stabilized, I would be fine with changing to a terser message, perhaps with just a link to the Cargo book.

Just incorporated your suggestions @weihanglo.

Thanks!
@bors r+

:pushpin: Commit 1c4065c52e6a2b81186a4a8b8a6b9e2134f10357 has been approved by weihanglo
It is now in the queue for this repository.

:hourglass: Testing commit 1c4065c52e6a2b81186a4a8b8a6b9e2134f10357 with merge 2a4a9b48fbcde2d1110ebac026176dcfdbb7104e...

:sunny: Test successful - checks-actions
Approved by: weihanglo
Pushing 2a4a9b48fbcde2d1110ebac026176dcfdbb7104e to master...


No license

Can PR the license additions if you tell which one you'd like :)

That would be awesome! We want the same as on Apollo Client. You could also add an authors file at the same time, if you like. 🙂


Conditional Layer

Can you elaborate a bit on the usage of Conditional GRU/LSTM over a regular GRU/LSTM ? What are the advantages and disadvantages of it ?
P.S
Also Adding convolution layers with filter sizes can lead to better generalizations  during decoding

A conditional GRU/LSTM consists in two GRU/LSTM activations  with an attention mechanism in between.
The first GRU/LSTM  computes an intermediate hidden state according to the previous output. Next, the attention mechanism is applied for weighting the annotations of the encoder with respect to it, computing a context vector. This context vector is the input to the second GRU/LSTM. See page 2 or Sec 3 from these papers.
The advantages are that usually conditional units perform better than regular ones.
The disadvatages are that conditional units  have more parameters and computational complexity than regular ones.
Overall, I think the usage of conditional units pays off.

Thanks for the clarification !


Headers cannot be returned from a DO WebSocket response

Headers specified in the WebSocket response returned from the fetch function of a Durable Object appear to be ignored in miniflare but not in production. Here's a repro.
In this case, the x-it-worked header is returned when deployed to Cloudflare, but not in miniflare.

Not sure, but perhaps this is the reason?

Awesome, thanks @mrbbot!