Autorest.csharp not found

Trying to generate csharp code using --legacy and autorest fails attempting to download any dependencies.
Help would be much appreciated.
AutoRest code generation utility [cli version: 3.6.3; node: v18.15.0]
(C) 2018 Microsoft Corporation.
https://aka.ms/autorest
info    | AutoRest core version selected from configuration: ~2.0.4413.
Loading AutoRest core      'C:\Users\user.autorest@microsoft.azureautorest-core<EMAIL_ADDRESS>(2.0.4421)
Installing AutoRest extension<EMAIL_ADDRESS>(~2.3.79)
Failed to install or start extension<EMAIL_ADDRESS>(~2.3.79)
Error: Package<EMAIL_ADDRESS>- '2.3.82' failed to install:
Failed to install package<EMAIL_ADDRESS>-- Error: Process Failed.Error: Failed to install package<EMAIL_ADDRESS>-- Error: Process Failed.
at install (/node_modules/@microsoft.azure/extension/dist/main.js:303:15)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async ExtensionManager.installPackage (/node_modules/@microsoft.azure/extension/dist/main.js:472:13)
at async Configuration.CreateView (C:\Users\user.autorest@microsoft.azureautorest-core@2.0.4421\nodemodules@microsoft.azure\autorest-core\dist\lib\configuration.js:685:51)
at async AutoRest.RegenerateView (C:\Users\user.autorest@microsoft.azureautorest-core@2.0.4421\nodemodules@microsoft.azure\autorest-core\dist\lib\autorest-core.js:55:29)
at async currentMain (C:\Users\user.autorest@microsoft.azureautorest-core@2.0.4421\nodemodules@microsoft.azure\autorest-core\dist\app.js:303:20)
at async mainImpl (C:\Users\user.autorest@microsoft.azureautorest-core@2.0.4421\nodemodules@microsoft.azure\autorest-core\dist\app.js:506:20)
at async main (C:\Users\user.autorest@microsoft.azureautorest-core@2.0.4421\nodemodules@microsoft.azure\autorest-core\dist\app.js:530:20)

I was able independently install the missing csharp extension but the Autorest --legacy command does not find the extension I just installed and continues to attempt to download it and fails.

This seems to be tied to the use of the --legacy. As soon as I remove the flag points to the latest csharp extension and succeeds. Are the csharp 2.3.* extension versions still available? I can get them using >npm install -g<EMAIL_ADDRESS>fine enough.

Tried without --legacy:
AutoRest --output-folder=./V20210201 --input-file=......\src\Microsoft.Azure.Devices.Registration.Gateway\Swagger\2021-02-01-preview\service.json --csharp --add-credentials --namespace=Microsoft.Azure.Devices.Registration.Clients.Service.V20210201
AutoRest code generation utility [cli version: 3.6.3; node: v18.15.0]
(C) 2018 Microsoft Corporation.
https://aka.ms/autorest
info    | AutoRest core version selected from configuration: ^3.7.0.
info    |    Loading AutoRest core      'C:\Users\user.autorest@autorestcore@3.9.4\nodemodules@autorest\core\dist' (3.9.4)
info    |    Loading AutoRest extension '@autorest/csharp' (latest->3.0.0-beta.20230309.2)
info    |    Loading AutoRest extension '@autorest/modelerfour' (4.25.0->4.25.0)
fatal   | Process() cancelled due to failure
error   |   Error: Could not read 'file:///C:/Users/user/src/Microsoft.Azure.Devices.Registration.Gateway/Swagger/2021-02-01-preview/service.json'.
error   | Autorest completed with an error. If you think the error message is unclear, or is a bug, please declare an issues at https://github.com/Azure/autorest/issues with the error message you are seeing.

Once I patched the command and used an explicit address to the missing file I got the error below. I suppose this is because our swagger files are not ready for the V3 generation.
AutoRest --output-folder=./V20210201 --input-file=C:\Users\user\dps\src\Microsoft.Azure.Devices.Registration.Gateway\Swagger\2021-02-01-preview\service.json --csharp --add-credentials --namespace=Microsoft.Azure.Devices.Registration.Clients.Service.V20210201
AutoRest code generation utility [cli version: 3.6.3; node: v18.15.0]
(C) 2018 Microsoft Corporation.
https://aka.ms/autorest
info    | AutoRest core version selected from configuration: ^3.7.0.
info    |    Loading AutoRest core      'C:\Users\user.autorest@autorestcore@3.9.4\nodemodules@autorest\core\dist' (3.9.4)
info    |    Loading AutoRest extension '@autorest/csharp' (latest->3.0.0-beta.20230309.2)
info    |    Loading AutoRest extension '@autorest/modelerfour' (4.25.0->4.25.0)
fatal   | Process() cancelled due to failure
error   |   Error: Operation response '/paths/~1deviceGroups~1{deviceGroupId}~1deviceRecords~1{deviceId}~1deviceTransfer/delete/responses/default' produces type couldn't be resolved. Operation is probably is missing a produces field and there isn't any global value. Please add "produces": []"
error   | Autorest completed with an error. If you think the error message is unclear, or is a bug, please declare an issues at https://github.com/Azure/autorest/issues with the error message you are seeing.

Hi @Selekena first I need to emphasize that the track 1 generator is officially deprecated and no longer supported, and we really cannot help much around it.
But for your case, you could try to remove everything in C:/users/<your user name>/.autorest which is the local cache of autorest when it installs its extensions. Running autorest --reset should also work I assume.
I never heard that we have taken off any legacy versions out of npm therefore I do not think it is because the version is not there.
Or you could try to run the same command in a fresh new environment to see if anything is conflicting on your side (for instance in a VM)

Assign this to @HarveyLink  since this is about track 1 generator

Usually for Track1 SDK, we use autorest V2 version, no --legacy flag was used in our process.
Here is a sample running command we used: autorest https://github.com/Azure/azure-rest-api-specs/blob/main/specification/netapp/resource-manager/readme.md --csharp --version=v2 --reflect-api-versions --use<EMAIL_ADDRESS>--csharp-sdks-folder=<folder>


Revert "Revert "Uncouple Mailjet from the application""

This reverts commit 4df05c369159d7133aa3c69643b91f7d21f6e012.

note: this should be merged once #1555 is ready-to-merge.


Add question help text to options

How to check for each option
Is there a way to do so in case.net?
https://github.com/nonprofittechy/docassemble-MOHUDEvictionProject/blob/7b0a52ed846fb0abebac7b600ca755705f422ec7/docassemble/MOHUDEvictionProject/data/questions/MOHUDEvictionProject.yml#LL1503C22-L1503C22

combined with #216


fix(android): custom maker performance improvements when view tracking

This issue was identified by profiling the app when the performance dropped to 10FPS.
Does any other open PR do the same thing?
No
What issue is this PR fixing?
https://github.com/react-native-maps/react-native-maps/issues/4968
How did you test this PR?
This change was tested on a Pixel 7 Pro using the sample code from the attached issue. With this change in place adding 20 markers sequentially has no performance impact and the app could maintain 120fps. ( Without the change the FPS dropped to about 10 ).
I also added some code to change the contents of the custom view and verified that it was updating on the map.

@jan-kozinski @mateki0 can you guys please take the time to review / test

I think it works as it should. Without this fix adding about 25 markers caused fps drop to ~1 and sometimes map is freezed on my device. With it it still lose some fps but it's much better and map is usable. I have also checked other Marker related examples and it looks like everything is fine (Animated marker position, Postion marker wtih z index, Massive custom markers).


Create list of needed identifiers and definitions

Working doc for sorting our unique NMDC id shoulders : https://docs.google.com/spreadsheets/d/1HdY4PZSKJzmWsUIzgmsX6FxaS9k1Jxt-IppRkoyhtBA/edit?usp=sharing
Working doc for types of identifiers needed: https://docs.google.com/spreadsheets/d/1e7hA0H4lkkHCNjYxMSkTR3BrYI3Jp-lJsnfOGyK28L8/edit?usp=sharing

@mslarae13 moving to Sept sprint but if you plan to not work on it in the next 2 weeks let me know so I can remove and add the backlog label

It's not enough to list something because it is needed.
Proposals for any new identifiers in the schema should include:

to what class will the identifier apply?
is the identifier required for any instance of the class above? or recommended, or optional?
should the slot allow multiple identifier values, or just one?
does it link to something else modeled by the schema, or does it link to something outside of the schema?
a few examples of valid identifier values
a few examples of invalid identifier values
is it a subslot of an existing slot? (I can help determine that.)

It's not enough to list something because it is needed.
Proposals for any new identifiers in the schema should include:

to what class will the identifier apply?
is the identifier required for any instance of the class above? or recommended, or optional?
should the slot allow multiple identifier values, or just one?
does it link to something else modeled by the schema, or does it link to something outside of the schema?
a few examples of valid identifier values
a few examples of invalid identifier values
is it a subslot of an existing slot? (I can help determine that.)

Can this criteria be a template for these types of identifier requests?

Checked in with @turbomam and moving this to the squad sprint.  @mslarae13 let me know if you're not planning to work on this the rest of this month.

my current thoughts on candidate typecodes for NMDC schema entities/classes:
sa - sample
st - study
saa - sample activity (e.g. sample prep) (range: sample)
p - party (e.g. instrument, person, software agent)
op - omics processing (workflow)
opa - omics processing (aka "data production"?) activity (domain: sample, range: data object)
do - data object
oaa - omics analysis activity (domain: data object. range: data object)
Note there are no separate mta, mga, etc. typecodes. These are all just oaa. For this squad, it's most important to assign study and (bio)sample codes while thinking about how they might fit in a larger scheme, without necessarily deciding on any other codes definitively at this time.

/cc @pkalita-lbl

@mslarae13 @sujaypatil96 Can this issue be closed today or by Tuesday?

What identifiers are needed has been curated. : https://lucid.app/lucidchart/811029cb-a93e-436a-a997-3faba459497c/edit?viewport_loc=-1802%2C-1249%2C4975%2C2700%2C0_0&invitationId=inv_46ca52cd-6253-4c5d-a913-e8d6df360a42#
I'll now start working on getting the info into the schema.


Colors as hex values

I have some favorite colors from themes and sports teams for which I know the exact hex values for the color. It would be neat if I could set the colors to those instead of eyeballing it using the wheel.

Thanks for the feature request. Unfortunately, the color picker library I'm using doesn't seem to support this out of the box. I don't think it would be very difficult to build, but I don't have much free time right now so I can't commit to doing it myself. PRs are welcome, of course! (either here or in the color picker library) Or if you can recommend a different color picker that has this functionality.


feat: match MAPPO performance

What?
Updates to current Jax MAPPO implementation to match the Tensorflow MAPPO implementation.
Why?
We firstly want to be able to compare the Jax system against the Tensorflow system fairly and secondly we want the Jax implementation to match and ultimately exceed the Tensorflow system's performance.
How?

Added flag for normalising advantages during training

Extra
[Any extra information.]

Linking a possible cause #628 (there is no masking of dead agents in value loss)


Rename session variable pipeline_profile_mode to pipeline_profile_level

What type of PR is this：

[ ] bug
[ ] feature
[ ] enhancement
[x] others

Which issues of this PR fixes ：

Problem Summary(Required) ：

Rename session variable pipeline_profile_mode to pipeline_profile_level

run starrocks_fe_unittest


spec: add clear(x) builtin, to clear map, zero content of slice

There is no way to clear a map in Go. You can write
for k := range m {
    delete(m, k)
}

but that only works if m does not contain any key values that contain NaNs.
Based on the discussion in #55002, we suggest adding delete(m) to the language to clear the map, always (even if it contains NaNs).
This is somewhat a reopening of #45328, which we closed because the loop was good enough, but we missed the earlier discussion in that issue of NaNs.
Adding delete(m) would then let us also add a similar mechanism in reflect.
We wouldn't need to add maps.Clear(m) since delete(m) is just as good.

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

Personally, I'm ok with it but can't determine if it is good enough.
Having a way to disallow NaN containing values in maps would be best. Then map clearing via iteration just works and the change in this proposal is unnecessary (if not for perf reasons).
Otherwise, comparing two maps may still have edge cases. (the crux of the issue is that comparison is actually underspecified it seems).
Notably relevant for Set types that would be implemented by using maps.
With this proposal, map clearing gets solved of but not inadvertent comparison of potential NaN composites.
If deemed a serious issue, I'm a bit concerned that it could hide a programming mistake.
Other than that, this proposal should be semantically sound at least.

I worry that this could lead to programmer error when clearing map[string]map[string]T or map[string]any.

With delete(m), will it be clear whether this is releasing memory of the underlying map?

If it does release the memory, it is equivalent to if m != nil { m = make(m[K]V) }. (But maybe I am not thinking through aliasing enough?)
If it does not release the memory, will users assume that it does and misuse it? My guess is yes. So if this goes forward with this semantics, maybe document that in the spec?

@atdiar

If deemed a serious issue, I'm a bit concerned that it could actually hide a programming mistake instead.

I think this proposal strictly improves the situation for irreflexive keys: if you use it it does the right thing for clearing the map, and if you don't use it you're no worse off than if it never existed.
That said, this still wouldn't help use-cases that need to add and remove individual keys: delete(m) would only give an all-or-nothing approach to deleting entries for irreflexive keys.

The programming mistake I'm more worried about is typos. If both delete(m, k) and delete(m) are defined, then I could inadvertently write delete(m) when I meant delete(m, k), and they're visually similar enough that the bug could be difficult to spot. 😅

With delete(m), will it be clear whether this is releasing memory of the underlying map?

I would assume this does not release the memory, just as the delete-with-a-for-loop doesn't.
If we ever implement #20135 the map will shrink eventually if it is never refilled.

@carlmjohnson To be clear, what's the programming error?

What @bcmills said, a typo of delete(m, k) to delete(m) that is easy to overlook.

a typo of delete(m, k) to delete(m) that is easy to overlook.

I'm not too worried about this. For one, any reasonable test would catch this mistake. Second, you're not actually using k, which the compiler might just catch for you (e.g. if the previous line was k := ... and no other use of k appears later). Third, this is not a problem unique to this case. Forgetting an argument can happen in all manner of ...-arg'ed functions: fmt.Printf("%d"), append(s), path.Join(p), and so on.

This proposal has been added to the active column of the proposals project and will now be reviewed at the weekly proposal review meetings. — rsc for the proposal review group

Hey! Are these proposal meetings open to public, by any chance? I would love to sit in on these meetings and learn.

@dheerajdlalwani They aren't, sorry.  (The general topic is better discussed on golang-dev than on this issue.  Thanks.)

While I understand the motivation to reuse a predeclared function name here, I'm concerned because delete(m) doesn't actually delete the map but removes all entries. I understand that clear(m) would create backward compatibility issues, but a maps.Clear(m),  reflect.ClearMap(m) or a runtime.ClearMap(m) would express the intention explicitly.

I understand that clear(m) would create backward compatibility issues

It wouldn't. All the builtin functions are predeclared identifiers. Adding new predeclared identifiers is backwards compatible - cf. comparable.
That's not saying we should (I have no strong feelings), but this, at least, is not a reason not to do it.

@bcmills

I think this proposal strictly improves the situation for irreflexive keys
at the cost of having an overloaded built-in function that behaves differently based on the number of arguments.

It's less user-friendly and more cognitively demanding to think about language features in terms of exceptions and/or gotchas, instead of just relying on different functions doing different things.
Yes, the language already has overloaded built-ins e.g. make but I feel the use case in this proposal is subtle enough to warrant a new built-in with a different name, e.g. clear(m) or reset(m).

If #20135 is implemented what would be the actual difference between
delete(m)

and
m = nil
m = make(map[type1]type2)

?

If https://github.com/golang/go/issues/20135 is implemented what would be the actual difference [...]

If this proposal is adopted as delete(m), and if #20135 is implemented without any new API, then it would depend on the heuristic(s) adopted in #20135, but I would guess delete(m) would clear the elements but leave the capacity.
There is some related conversation in a different issue #54454, including this comment from Keith in https://github.com/golang/go/issues/54454#issuecomment-1216112699

We probably don't want to shrink if people do the map clear idiom + reinsert a new set of stuff. Perhaps we only start shrinking after 2N operations, so that N delete + N add involves no shrinking. There's definitely a tradeoff here between responsiveness when you use fewer entries and extra work that needs to be done to regrow.

Perhaps #20135 and this proposal could be solved together (resize? remake?) or at least briefly considered together. (FWIW, I think some solution to #20135 probably has broader impact than this issue, though that probably is a better discussion for elsewhere).

If https://github.com/golang/go/issues/20135 is implemented what would be the actual difference between
delete(m)
and
m = nil // this is not actually needed
m = make(map[type1]type2)
?

The difference comes down to aliasing. What happens if other variables have a reference to the same map? See https://github.com/golang/go/issues/56351#issuecomment-1286121769

I really don't like that delete(m) does not, in fact, delete m. Please don't take this shortcut. It makes the code lie, and that is a grievous offense for something that would be built into the language.
Personally, I think that it's a bug that you can have values that aren't equatable as map keys, and changing the language to better handle problems caused by that bug is not a good idea. I understand we can't change the fact that NaN is a valid map key, but we don't have to perpetuate the error by adding features to make it more usable.
My first test for most languages features is "is this just trying to avoid writing a loop or if statement?" and if it is, I put it loooow on the list. If we ignore the NaN edge case, that's all this is.
I'm not sure I've ever cleared a map in production code (though I do understand the uses). I am pretty sure I've never used floats as keys in a map. Adding a language feature for a tiny edge case based on a bug seems like an easy "no".
That being said, if people reeeealllly want this, maybe make it delete(m, _) instead? At least then it's kinda clear you're not deleting m itself, and you're not having one method that changes behavior based on how many arguments you give it.

My first test for most languages features is "is this just trying to avoid writing a loop or if statement?" and if it is, I put it loooow on the list. If we ignore the NaN edge case, that's all this is.

FWIW one of the reasons this is being asked for now is that I wished to have a method in reflect which efficiently (!) clears a map. One of the main reasons for not adding that method to reflect is that reflect should not allow doing something the language can't do either.
So, for me, this isn't really about the syntactical overhead and effort of writing a loop. It's about being able to clear a map from reflect at all, without having to pay the allocation to iterate it first and then pay the allocations for all of its storage when growing it again.
I agree that it would be better if we could've done something about irreflexive keys before Go 1. But to me, the reason for this isn't really to be able to clear them out. That's such an edge-case, it doesn't even factor into the calculation for me. As I said here, I'd even want something like this if delete(m) didn't remove irreflexive keys. Though, TBF, maps.Clear couldn't use the fallback I could do with reflect.

I want to note that the proposed delete(m) is precisely analogous to the existing os/signal.Notify.  Notify(ch, sig) notifies for the signal sig, while Notify(ch) notifies for all signals.  Similarly, delete(m, k) deletes k from m, while delete(m) deletes all k values from m.
That analogy suggests that perhaps we should permit delete(m, k1, k2) to delete both k1 and k2 from the map.

I'd vastly prefer clear(m) to delete(m) if we need it for optimization's sake. Then it's not a misleading name, it matches the other Clear methods, there's no confusion about how many arguments it takes, and it's much more clear what it does. Heck, we could even allow it to also take in a slice and zero out the memory of its backing array. There's no easy way to do that right now, either.

@ianlancetaylor
I think I can understand why people would like to dissociate map clearing and delete.
In one case, everything is deleted regardless of the key.
In the other, NaN containing keys would not be deleted.
Is this an issue?

It seems to me that:

We would not care about delete if it were possible to delete irreflexive keys otherwise.
Deleting irreflexive keys from a map is a very niche use-case, because putting irreflexive keys in a map in the first place is a very niche use-case.

Given those, I wonder if it would suffice to define delete as the inverse of m[k] = v. That is, instead of having delete(m, k) remove all keys equal to k, we could have it delete all map entries that could have been created by assigning to m[k].
That would change the behavior of a tiny number of existing programs that both add irreflexive keys to maps and use delete with those keys. My hypothesis is that essentially every such program is already broken to begin with, and redefining delete in this way would strictly reduce their degree of brokenness.

@ianlancetaylor if you could delete multiple keys delete(m, keys...) would be useful but it would also be dangerous if len(keys) == 0 cleared the map.

If it does not release the memory, will users assume that it does and misuse it? My guess is yes.

What's the misuse? Semantics don't change. I am opposed to specify allocation behavior. That's an optimization that's up to implementers.

@Merovius Say they have a long lived struct and don't need a map field anymore so they want to free the underlying memory. The concern is a user writes delete(x.m) instead of x.m = nil here and is consuming more memory than they intended. My point was that the documentation for delete just needs to be written in a way that that users should not expect to get the memory back. If this can be done without specifying the allocation behavior, I agree that that would be preferable. I am just not sure how.

@bcmills isn't that equivalent to NaN == NaN for map keys? :-)
It's funny how we circle back to the same options as the precedent proposals.
Given that I think non reflexive map keys are always a mistake and that a proper solution, while possible, might be quite involving and not retroactive, I would be sympathetic to that.
I really fail to see any use-case for non-reflexive map keys.

@timothy-king FWIW one reason why I object to specifying any behavior about the memory behavior of delete(m) is that I also don't want to prevent the GC from freeing the storage of that map, if there is memory pressure. I don't think it's realistic that this would get implemented, but it could be and would be a sensible definition.
By analogy, this is what the spec says about the second argument to make for maps:

A new, empty map value is made using the built-in function make, which takes the map type and an optional capacity hint as arguments: […] The initial capacity does not bound its size: maps grow to accommodate the number of items stored in them

It says nothing about what it does, except that it's "a hint". In fact, what little it does say doesn't even make a lot of sense - maps don't have a "capacity", AFAIK.

Making delete(m, nan) remove all NaNs from m seems like a change likely to correct undetected bugs in the wild. Does anyone have access to a corpus to see how many existing Go programs use map[float64|float32]T and how?

how many existing Go programs use map[float64|float32]T and how?

float64 and float32 are not the only irreflexive key types. It also affects structs and arrays. I also don't think using such a map alone is a good indication, as most floats are reflexive.
If I would try to get data on this, I'd suggest patching the Go compiler to panic (or crash) on an irreflexive map key and run tests on a large Go corpus with that modified compiler. I think that patch should be relatively simple (as that is special-cased already for the hash calculation) and the Go team has access to a pretty large corpus, if they want to test that.

@atdiar

isn't that equivalent to NaN == NaN for map keys? :-)

No, for two reasons!

It would not change the insertion behavior. (k := math.NaN(); k[f] = true; k[f] = true would continue to insert two entries, as it does today.)

It would not equate NaNs with different representations, since a NaN with one representation cannot create an entry in the map for a NaN with a different representation.

I am not a fan of this proposal honestly. delete feels ambiguous considering a map can be nil too.

@bcmills
I see. I guess it will require to treat NaN containing keys specifically so that there is no need to iterate the whole map to get rid of keys with the same representation?

instead of having delete(m, k) remove all keys equal to k, we could have it delete all map entries that could have been created by assigning to m[k].

I find this very intriguing. However, it is worth noting that this makes deletion of a NaN key significantly more expensive. In the current implement, it might require iterating over the entire map.

It might be possible to use additional memory to track irreflexive keys so that delete(m, nan) does not require iterating over the map?
This already uses more memory than some people would expect, so maybe not terrible to make it use a bit more:
	m := make(map[float64]int)
	for i := 0; i < 1000000; i++ {
		m[math.NaN()] = 1
	}

Also, perhaps if done in a release in conjunction with something like #54766, the overall memory usage could drop in that release even for maps with many NaNs.

I don’t know how to track irreflexive keys without either penalizing all maps or adding significant complication and code weight. But perhaps it is possible.

If I would try to get data on this, I'd suggest patching the Go compiler to panic (or crash) on an irreflexive map key and run all tests on a large Go corpus with that modified compiler.

Alternatively, it could export a counter via runtime/metrics. That way you could even check if any production code uses it by having your monitoring scrape it. Hypothetically. No idea if Google ever does something like that, but it would be a fancy data collection tool for cases like this.

If delete(m) were instead delete(m, _) it might be clearer that it's flushing key/value pairs rather than a C++ delete of the map. In terms of violence to the spec, it would add another case to the uses of the blank identifier that is undeniably special, but arguably not more special than the other uses of the blank identifier. (It might be suggestive of a notion that a right-hand blank identifier is always equal to the left-hand operand, iff the left-hand is not nil - I think that's a sound formulation of the ailment w/r/t maps, not sure whether that would be more generally sound.)

If we were looking for alternative spellings for delete(m), I'd suggest delete(m, ...).

But I don't personally see either as much better than just delete(m). (Which, FWIW, I'm on the fence about.)

I agree with previous comments that clear(m) or similar, reveals the intent of what I as a developer I am thinking of.
Also IDEs will currently highlight errors if I want to delete a key but type delete(m) by mistake, by adding this proposal it could be misleading/introduce bugs as IDEs will probably not highlight as an error.
Yes as good devs we have tests etc :), but it is an easy typo to make

Leaving the default container behavior as is could be an option. Then perhaps that clear/delete would make sense.
In a potential future where constraints can be used as mere interface, we would probably want a new constraint/interface that defines the set of types that do not contain NaNs.
That constraint would be used for Set datastructures so as to avoid breaking operational semantics (such as cardinality computation) for instance.
We would then be able to assert against this interface. The point of the assertion is to avoid aliasing which could modify a float64 field before map insertion. Especially important wrt to concurrency.
I think it's close to being equivalent to using:
func ContainsNaN[T comparable](v T) (bool, T){
    if v! = v{
        return true, v
    }
    return false, v
}

but compiler-enforced.
 Needs a shallow unaliased copy.
Tl,dr:
We could adopt this proposal without changing the language if it only affects the basic map container. People may want to use more constraining wrappers when needed.
I think, NaN affects typestate. Subtyping/interfaces/constraints can probably be used to model typestate changes. The issue for typestate tracking to be tractable is to tame aliasing. That may be possible.
It's even more relevant because NaN implies interfaces cannot be provably reflexive in general.
Just thinking out loud. Might be some things I am overlooking.

I think it is easy for a container to add a if v != v { panic("irreflexive value inserted") } at the top of its methods. It's not type-safe, but for an edge-case like that, this seems good enough to me. Better than coming up with a new name for a new predeclared constrained (as it couldn't be built using type sets, similar to comparable) which then is even overly restrictive, as most floats are have perfectly reflexive equality.
That is, I'm not worried for containers, as they already are wrappers hiding the map and so can do additional checking. The maps package and users of reflect are different, though, as they operate on maps they don't own.

We can't change what is valid in a map, nor are we going to change map semantics regarding NaNs. That's incompatible and only moves us from one set of sharp edges to a different set. It's not better, just different. (More at https://research.swtch.com/randhash; look for "alternatives are worse".)

I spoke to @robpike for a while about this, and we both agree with @natefinch that this operation should be called clear(m), not delete(m). In other languages delete(m) is something else entirely, and it is telling that all previous discussions have used the word clear instead of delete when we describe the operation.

@robpike and I also agree with @natefinch that we should consider making clear(x) zero the backing store when x is a slice or array pointer, which I believe @griesemer also suggested in an earlier conversation.
Originally Rob and I had thought that clear(x) should zero x[:cap(x)], but on further reflection I think that would be a mistake, since it would make clear(x[:i]) no different from clear(x), which would be very confusing - you'd have to write clear(x[:i:i]) instead. If people want to clear to the capacity, it is straightforward (even clear!) to write clear(x[:cap(x)]).
So the definition of clear(x) would be that if x is a slice or a pointer to an array, then clear(x) is the same as
for i := range x { x[i] = 0 }

(I am posting separate comments so people can thumbs up/thumbs down these comments individually.)

How about clear(aChannel)? Equivalent to for range aChannel {}?

One more thought about clear(x): perhaps instead of pointer-to-array being special, clear should zero pointer-to-anything, so that if x is a *T, then clear(x) is the same as *x = zero-value-of-T.

I'd like to push back against this. We only need clear for maps because it does something that isn't possible to do in the language (clear NaN keys). For pointers and arrays, that isn't true. In fact, you can already write that function using generics.
func zeroPointer[T any](p *T) {
	var zero T
	*p = zero
}

Let's not add a second way to do things that nobody asked for.

How about clear(aChannel)? Equivalent to for range aChannel {}?

No. Clear is not about communication.

I'd like to push back against this. We only need clear for maps because it does something that isn't possible to do in the language (clear NaN keys). For pointers and arrays, that isn't true. In fact, you can already write that function using generics.  Let's not add a second way to do things that nobody asked for.

That last bit seems a bit uncalled for. Let's keep this respectful.
The rationale "we don't need something convenient because we already have a different way to do it" is not how to approach the decision. If we used that as a hard rule, we wouldn't have 'for' at all, since we already have 'goto'.
Instead, I would suggest using at least the following criteria:

Does the change make anything possible that is impossible today? (Yes, clearing maps for real.)
Does the change fit well into Go as a whole, seeming like a general, orthogonal concept, or does it seem like a special case? (If clear handles all our pointer-like data, that seems more general than just maps.)
Does the change improve the overall software engineering experience of using Go? (Yes, because clear(x) is, well, clearer and simpler than the alternatives.)

I do agree that when there are multiple ways to do something, you have to have a good answer for when to choose each. But at the same time I don't see many people being confused about which of these they should write:
clear(x)

for k := range m {
    delete(m, k)
}

for i := range x {
    x[i] = *new(T)
}

*x = *new(T)

Clear expresses the purpose far better than any of the others. I just can't imagine anyone arguing that you should use a for loop or an explicit assignment in preference to clear(x). Also, if you care about zeroing things to enable garbage collection, the compiler may well optimize away some of the other forms, but it would guarantee not to optimize away clear.

My main concern with throwing pointers into the mix is that it makes clear less clear.
For example, If i see clear(&foo.buffer) in a codebase, what does it do? if foo.buffer is an array [123]int, it clears the array. If foo.buffer is a slice []int it (presumably) clears the slice header but leaves the backing array alone. Is that what the author intended or is it a bug? Hard to say.
If clear only works on maps then it is clear what clear(foo) does: foo must be a map, and it clears that map. If foo is a pointer to a map (er, does anyone use pointers to maps?) it is a compile error, not a subtle bug.

At the moment we don't have a general way to express the zero value of a type parameter. We currently suggest that people write something like
var zero P // P is a type parameter
x = zero   // set x to the zero value for P

I'm slightly concerned that people start writing
clear(&x)

for this case if clear is available in this general form. This seems a bit misleading.
Perhaps clear should be restricted to maps and slices for a start (and perhaps pointer to array but I am not sure it's important).

OK, I'm convinced we should limit clear to map, slice, and possibly ptr-to-array. Good points everyone. Thanks.

Should it work for array, reset all elements to zero value when clear it?

Go one step further, should it can clear partial content ?
For slice, can we pass range to it, like: clear(slice, x, y)

@leaxoy If you want to clear a subrange of a slice, just do clear(slice[x:y]).

Instead of introduce new global function, is the introduction method more appropriate?
Then this becomes to slice.Clear() or map.Clear().
Or more aggressive, add builtin interface. User can define their's custom clear action for arbitrary type.
type Clearable interface {
    Clear()
}
// or
type Resetable interface {
    Reset()
}

@leaxoy, it can't work for an array: if you have
var x [10]byte
clear(x)
The call clear(x) should be passing a copy of x to clear. If clear can modify the original x, it's not a real function.

We talked about defining maps.Clear in #55002, but how? It would not be possible to write in pure Go code. Instead it would need some kind of hidden entry point into the runtime.
In the past I've referred to that kind of library change as a "back-door language change", meaning it's a language change - it adds something not possible in pure Go - but pretends not to be one. We try hard to avoid doing that.

Ok, I will look at it and understand the context.

Discussion seems to have trailed off, so it sounds like people are in favor of calling the builtin clear(x), and it would apply to maps (deleting all entries, even NaN-containing ones), slices (zeroing all elements up to len, not up to cap), and pointers to arrays (zeroing all elements).
It's true that clear(x) changes len of a map but not len of a slice/array ptr, but that's true of other operations too (like assigning to x[i]).

We also need to say what happens for arguments of type parameter type. It seems straight-forward to accept all type sets with underlying types that are one of map, slice, or pointer to array.
Tentative spec wording:

The built-in function clear takes an argument of map, slice, pointer-to-array, or type parameter type. For maps, clear deletes all entries, resulting in an empty map. For slices and arrays, clear sets all elements up to the length of the slice or array to the zero value of the respective element type. If the argument type is a type parameter, the type parameter's type set must contain only maps, slices, or pointer-to-array types, and clear performs the operation implied by the actual type argument.

A final observation: As is, clear([]int{1, 2, 3}) will be valid but meaningless. One could require the argument to be addressable. I am not suggesting we should (we currently also allow delete(map[int]int{1: 1}, 1)), just bringing it up for completeness.

@griesemer

If the argument type is a type parameter, the type parameter's type set must contain only maps, slices, or pointer-to-array types

This wording seems to imply that a value of type parameter whose constraint includes a union type element with diverse type terms can also be supplied to clear, i.e. the following should compile:
func Foo[T ~map[string]int | []string](v T) {
    clear(v)
}

Is this the intent?
If it's not the intent, perhaps it's better to refer to the type parameter's core type instead, like make:

The built-in function make takes a type T, optionally followed by a type-specific list of expressions. The core type of T must be a slice, map or channel.

Therefore let me propose:

If the argument type is a type parameter, the type parameter's core type must be a map, slice, or pointer to array, and clear performs the operation implied by the actual type argument.

The (proposed) intent is to allow the former (no need to restrict to core type). But depending on how that influences the implementation, we may start with the restriction to core types.

Something to be aware of: clear applied to slices and arrays permits the creation of zero values (elements) of possibly unexported types (in a package that doesn't have name access to the respective type and thus doesn't have a direct way to create a zero value of such a type). Per discussion in #56669, this is not adding a new capability to the language (but it does make it easier to create such zero values).

Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group

Clearing maps and slices makes sense to me. I'm not sure why clear needs to support clearing a pointer to an array; can't this be done by slicing the array?
var a [3]int
clear(a[:])

We can look at the existing special provisions made for pointers-to-arrays in the language.
Pointers-to-arrays are treated specially (like a kind of fixed-length slice) in a few places:

slice expressions
type conversions (as of Go 1.17 at least, where you can do (*[3]int)(aSlice))
for...range statements
The len and cap (pseudo)functions

However, they are not treated specially by append and copy today:
s := []int{1, 2}
a := [2]int{3, 4}
_ = append(s, (&a)...) // doesn't work
copy(s, &a)            // doesn't work

(The copy case is interesting, because there is a (very) special case there where if the slice is a []byte, you can use a string.)
If we think of clear as more akin to append and copy, then perhaps it makes sense to not include pointer-to-array support. (Or maybe we should go the other direction and make these builtins, or at least copy, support pointers-to-arrays. Has this been discussed before? Surely yes, but I can't locate the issue.)
OTOH, append and copy do not work on maps, so maybe clear is more like len/cap, which operate on a variety of types. But this analogy doesn't go very far because len/cap also work on strings and chans which clear will not accept.
I think my slightly preferred option is for clear to only support maps and slices initially, and we can separately discuss having it support pointers-to-arrays (perhaps in conjunction with copy).

I agree it makes sense to start with just supporting maps and slices and
defer the decision on pointers-to-arrays.
On Fri, Nov 18, 2022 at 4:07 PM Caleb Spare @.***>
wrote:

We can look at the existing special provisions made for pointers-to-arrays
in the language.
Pointers-to-arrays are treated specially (like a kind of fixed-length
slice) in a few places:

slice expressions
type conversions (as of Go 1.17 at least, where you can do
(*[3]int)(aSlice))
for...range statements
The len and cap (pseudo)functions

However, they are not treated specially by append and copy today:
s := []int{1, 2}
a := [2]int{3, 4}
_ = append(s, (&a)...) // doesn't work
copy(s, &a)            // doesn't work
(The copy case is interesting, because there is a (very) special case
there where if the slice is a []byte, you can use a string.)
If we think of clear as more akin to append and copy, then perhaps it
makes sense to not include pointer-to-array support. (Or maybe we should go
the other direction and make these builtins, or at least copy, support
pointers-to-arrays. Has this been discussed before? Surely yes, but I can't
locate the issue.)
OTOH, append and copy do not work on maps, so maybe clear is more like len
/cap, which operate on a variety of types. But this analogy doesn't go
very far because len/cap also work on strings and chans which clear will
not accept.
I think my slightly preferred option is for clear to only support maps
and slices initially, and we can separately discuss having it support
pointers-to-arrays (perhaps in conjunction with copy).
—
Reply to this email directly, view it on GitHub
https://github.com/golang/go/issues/56351#issuecomment-1320522639, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ACKIVXK43NHD4GB5G6CGAPDWI7VX5ANCNFSM6AAAAAARKMKMZA
.
You are receiving this because you commented.Message ID:
@.***>

Have you considered panicking when trying to assign a NaN float as a key to a map?
It could be one of those features that only activates if go.mod is at 1.21 .
Is there a real use case for when a NaN map key isn't a bug?

@thomasf See #20660. And to reiterate: The motivation here is not to be able to remove NaN keys from maps. It's to be able to efficiently clear them.

My first test for most languages features is "is this just trying to avoid writing a loop or if statement?" and if it is, I put it loooow on the list. If we ignore the NaN edge case, that's all this is.

Hey @natefinch, how about ignoring the NaN edge case here and returning the fixed hash value just like any other floating number?
hashCode implementation of Double in the Kotlin/Java returns a predefined constant value (see hashCode and doubleToLongBits below). Maybe something similar can be implemented in go, though I have to mention that the memhash function returns a fixed hash value for NaN anyway. So maybe there is no need to handle it at all.
I tried to remove case f != f: case from f64hash and recompiled go but it didn't work even though f64hash indeed gets called when I created a map and inserted some key-values. It was as if the return value of f64hash function was irrelevant. I presume some underlying implementation of f64hash needs to be changed somewhere else(???). Any help here is appreciated.

@maykinayki See this comment regarding that suggestion.

@maykinayki See this comment regarding that suggestion.

The research shared in this comment is quite misleading. Iteration condition  i < 20 is chosen wisely in the equivalent Go program as the program exists in the 19th iteration even if you set the condition to, for example, i < 26 (see: https://go.dev/play/p/gpj4X9TCNbP). Try it locally, you will see that it does not run in linear time at all. So I don't see any optimization here in the hash function for NaN cases.

@maykinayki I don't understand what you are arguing. This is what your program prints on my laptop:
0 1 823ns
1 2 425ns
2 4 454ns
3 8 9.456µs
4 16 5.241µs
5 32 18.713µs
6 64 15.894µs
7 128 37.662µs
8 256 63.915µs
9 512 94.165µs
10 1024 173.128µs
11 2048 322.786µs
12 4096 681.056µs
13 8192 1.319906ms
14 16384 2.481955ms
15 32768 5.360443ms
16 65536 12.040117ms
17 131072 23.937675ms
18 262144 50.91205ms
19 524288 113.154386ms
20 1048576 235.864912ms
21 2097152 465.504537ms
22 4194304 1.043067046s
23 8388608 2.232165622s
24 16777216 5.043666817s
25 33554432 9.841919012s
26 67108864 26.609594195s

That looks pretty linear to me (ignoring the noise around the ends). But that's off-topic anyways. Whether or not Go's builtin hash function randomizes NaN keys is immaterial. The interesting part of that post is the "the alternatives are worse" section. We are not going to change the behavior of irreflexive keys for maps.

@Merovius, the post says that returning a random hash for NaN improves the performance, I argue that it has a bad performance anyway. Try to run the same program with a float value other than NaN, it finishes in a second. So where is the performance improvement for NaN? Whereas the same program in Kotlin returns similar execution times for NaN and any other float value.

@maykinayki The performance improvement comes from an asymptotically linear behavior vs. an asymptotically quadratic behavior. That matters a lot, when you talk about attacker-chosen map keys. Even if there is an overhead to using NaN, as long as that overhead is proportional to the length of the input data, there is no amplification effect.

My point is if there is no performance improvement on returning a random hash for NaN then let's return a fixed hash for it. So a map can have only a single NaN key which is readable

The post mentions this possibility and explains why we are not going to do it.

and removable thus no need to introduce a new built-in such as clear.

Again, the point of clear is not to be able to remove irreflexive keys. The point is to have an efficient way to clear maps.

@thomasf See #20660. And to reiterate: The motivation here is not to be able to remove NaN keys from maps. It's to be able to efficiently clear them.

Oh, I didn't think about the reflect package since I almost never use it directly myself.
In all other scenarios Go could just start using the same efficient compiler replacement that it already uses for all keys that does not contain floats ( https://go.dev/doc/go1.11#performance-compiler ) if NaNs were disallowed as keys.
Maybe that's not enough?

@cespare, thanks for pointing out append and copy. Removing ptr-to-array.

One advantage to clear (about which I am otherwise kind of skeptical/meh): There's no particularly good way that I see to write an optimized memclear idiom in generic code, because there's no generic literal zero.
There are options. @Merovius offline suggests *new(T) or doing enough data flow analysis to make var zero T usable.
But having clear neatly fixes the problem, with no weird code and no compiler shenanigans.

No change in consensus, so accepted. 🎉
This issue now tracks the work of implementing the proposal.
— rsc for the proposal review group

Just revisiting this, wanted to note that clearing slices might be a bit more involving.
If at some point, we want to make sure statically that some slices of interface values always contain non-nil values, things are going to be somewhat less tractable.
Reason being that slices alias the backing array and clear mutates it. (similar issue to typestate analysis in presence of aliasing)
Thinking about it, and if I'm not mistaken, I might be inclined to intoduce clear for maps only, at least at first.
Just a thought.

I think it is easier to statically check that N ==0 for such types.
However dealing with slice clearing is more difficult.

I don't understand what you mean. It does not matter in which way we special case make for those types. The point is that we need to special case those types for make anyways, so we can special case them for clear as well.

To ask a clear question: If T does not have a valid zero value, why doesn't "you can't call clear on a []T" address whatever problem you are apparently seeing?

Oh ok I see what you mean.
I wanted to avoid having to introduce that difference in generic code but regardless,  that difference would have to exist for make, so I guess it would have to exist for clear.

With a clear operating on []T I can make a clear working on *T by just doing clear(unsafe.Slice(p, 1)).
Not arguing that we should change the plan. Just observing the Turing-completeness of this fun language...

@randall77 That's true but I'm not sure what problem you are implying.
You can already write *p = *new(T) and/or implement clear on pointers that way. And I think if someone is willing to write clear(unsafe.Slice(p, 1)), they already opted out of clarity for their code anyways - which was the argument against adding clear for pointers AIUI.

No problem really. Just pointing out that clear not working on pointers directly doesn't mean it can't be used indirectly to do exactly that.
I agree that readability/confusion between clear(x) and clear(&x) is a good reason to not allow *T.

I dont see this mentioned yet, so let me do so now:

If the argument type is a type parameter, all types in its type set must be maps or slices, and clear performs the operation corresponding to the actual type argument.

https://tip.golang.org/ref/spec#Clear
this does not make sense, at least to me. clear can only operate on values, not types. what is the point of clear accepting a type parameter?

@4cq2 "Argument type" in this context means "the type of the argument" not "the type given as argument" (that is called a "type argument" in the spec. Isn't language fun?). I guess that could be made explicit, to remove that ambiguity?

@Merovius  right now its unclear to me (no pun intended). when I see the words "type parameter", I think about this:
package hello

import "fmt"

func world[T any](value T) {
   fmt.Println(value)
}

in which case T is the type parameter. so the documentation is instructing users to do:
clear(T)

which again makes no sense.

Yeah, happy to use a more lucky formulation that is not too clunky. But the meaning here is, in italics:
If the argument type (the type of the argument provided to clear) is a type parameter (is of type parameter type), all types in its type set (in the type set of the constraint corresponding to the type parameter) must be maps or slices, and clear performs the operation corresponding to the actual type argument (corresponding to the type of the actual type argument with which the type parameter was instantiated).

seems like it would be better to just say something like:

if the type of the argument is generic, then blah blah

@griesemer I think I would definitely replace "argument type" with "type of the argument" or introduce a name for it ("clear(x), where x has type T, does… if T is a type parameter…").

Reopening so we can reconsider.

it takes very little code to demonstrate the case and that could be used to introduce a good name to refer to:
func genericUseOfClear[T TypeSet](v T) {
  clear(v)
}

very much agree with @jimmyfrasche. the wording from @Merovius is a big improvement, but a small example is even better for dummies like me :)

Spec prose was adjusted. Closing.


[HELP] Cannot typing password sudo for WSL when initialize project using Laravel Sail

Laravel Version
11.9
PHP Version
8.3.12
Database Driver & Version
MySQL 8.0.30 for Windows (Laragon)
Description
When I want to start a new project using the command:
curl -s "https://laravel.build/super-apps" | bash
The installation process went smoothly but when the final adjustment by Laravel Sail needed to enter the sudo password from my WSL Ubuntu, I couldn't type or press the enter key. Just stuck on this process

When I canceled the command (when entering password sudo), I couldn't use the command: ./vendor/bin/sail up to run the laravel apps, but what appeared was the sail code file in my VSCode.

Steps To Reproduce

Initiate a new project like this : curl -s "https://laravel.build/super-apps" | bash
Wait until you should provide password root for WSL

Then enter the password sudo (but cannot do it just stuck on this process)

I drop this issue on https://github.com/laravel/sail/issues/739 but no one replying to me, i hope in this repo someone can help me

You do need Laravel Sail to be in your WSL environment.


Install pandas-stubs as a dependency if pandas is used

I think a few libraries that are checked by mypy-primer use pandas. It might be nice to install pandas-stubs as a dependency for those projects.
Background: It might be nice if pandas-stubs would use mypy-primer in the (far?) future. Especially if pandas-stubs is promoted by mypy https://github.com/python/mypy/issues/14328 I have absolutely no idea how difficult it would be to integrate a mypy-primer run for pandas-stubs PRs (ideally it would only need to be run on projects that actually use pandas) but I think the first step is to add pandas-stubs as a dependency :)

Thanks for the suggestion! I think I'd prefer to try to stick to type checking projects as they type check themselves for the per-project dependencies.
The good news though is that this doesn't affect your ability to use mypy_primer in pandas-stubs CI. You can simply put pandas-stubs in MYPYPATH and mypy_primer will forward it along to mypy which will pick it up, and will have precedence over anything installed. This is how we do things when testing third party stubs in typeshed, for instance.
It should be pretty easy to setup. Feel free to tag me on a PR / when I get time I will eventually make one myself.

@Dr-Irv What is your opinion of having a typeshed-like mypy-primer bot for pandas-stubs PRs: bot comments on a PR to indicate how the PR changes the mypy output on various projects?

Not quite sure I understand what you're proposing, but I think what you're suggesting is that if someone does a PR for pandas-stubs, the bot runs mypy (mypy-primer?) on projects that use pandas-stubs and sees if the PR would change the output for those projects.
If that's the concept, I'm fine with that, although concerned it would increase our CI time, and also the time it takes to review PRs because if the bot picked something up, we'd have to figure out what the change did that caused the issue, whether the issue was the stubs, or if it was invalid use of pandas from the outside project.

if someone does a PR for pandas-stubs, the bot runs mypy (mypy-primer?) on projects that use pandas-stubs and sees if the PR would change the output for those projects.

Yes, that is the idea.

although concerned it would increase our CI time

I'm not sure how long it would take. Hopefully, it could be run on a small(er) subset of projects.

and also the time it takes to review PRs because if the bot picked something up, we'd have to figure out what the change did that caused the issue

I think it would be beneficial to catch accidental changes. I can definitely see that when we tighten some arguments that it would trigger new errors (which could take some time to understand which are (in)valid).

mypy_primer runs in like 8m on typeshed and that's with a much larger set of projects than you'd need.

increase the time it takes to review PRs because if the bot picked something up

You can always just ignore it, which is equivalent to what you're doing today ;-)

You can always just ignore it, which is equivalent to what you're doing today ;-)

Ignoring what's invisible is easy.  Ignoring what's visible is much harder!

You can simply put pandas-stubs in MYPYPATH and mypy_primer will forward it along to mypy which will pick it up, and will have precedence over anything installed.

I'm locally playing around with mypy-primer and pandas-stubs but surprisingly it doesn't show me any changes:
git clone -b main --depth 1 git@github.com:pandas-dev/pandas-stubs.git
cp -r pandas-stubs pandas-stubs-new
rm -rf pandas-stubs-new/pandas-stubs/core/  # should cause some changes
python mypy_primer.py --new 1.1.1 --old 1.1.1 -k spark -o diff --old-mypypath pandas-stubs --new-mypypath pandas-stubs-new


Chanakya link updated

All Submissions:

[ ] Have you followed the guidelines in our Contributing document?

New Feature Submissions:

[ ] Does your submission pass build tests locally?
[ ] Have you indented your code locally prior to submission?

Changes to Core Features:

[ ] Have you added an explanation of what your changes do and why you'd like us to include them?
[ ] Have you successfully ran tests with your changes locally?

Hope that you have followed the contribution guidelines. Please don't make a request anywhere to get a review. One of the maintainers will get back to it within a day or so. 😃
(Powered by GitMate.io)

please review now

@rahulotwani I can still see changes in  README.md made here 😄

@rahulotwani : Resolve the conflicts and squash your commits


Map constructor does not expand variables

(let ((foo 'bar)) (map foo 'bar))

leads to
exception error: #(illegal_mapkey foo)

It should however return #{bar=>bar}.

IIRC this is a known issue on certain versions of OTP. For reference, which version are you on?

19.3
Erlang/OTP 19 [erts-8.3] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]

Hmm.. I want to see it was 19.1.x. Also, I'm on 19.3 with LFE v1.2.1 and I get the following:
lfe> (let ((foo 'bar)) (map foo 'bar))
#M(bar bar)

Strangely, I’m not able to reproduce the bug now, neither on develop (  22b30a15af861b680d1ae607252195c384a7682a ) nor on v1.2.1 and neither on 19.3 nor on 18.x. Also it looks like map values in REPL have now correct form #M(bar bar) instead of the one I reported, #{bar=>bar}. Closing the issue then, assuming it has been some version issue on my side.


[BUG] ESP32/M5 board crash loops - assert failed: xQueueTakeMutexRecursive queue.c:717 (pxMutex)

Reproduction steps
I flashed my M5 board with Matter SHA - bba4182d61d56510f628307aff9844b75fb27b0d. I'm seeing the board crash loop 100% after flashing the board.
I'm using ESP IDF 3.3.2.
Note: This does not happen with Matter SHA 9fbcf1cef8c4a42761a7fc23588ca71d75904ff5
assert failed: xQueueTakeMutexRecursive queue.c:717 (pxMutex)
Backtrace:0x40081e72:0x3ffe9c000x40092639:0x3ffe9c20 0x40098ea1:0x3ffe9c40 0x400936ee:0x3ffe9d60 0x401067c5:0x3ffe9d80 0x401069db:0x3ffe9da0 0x400f6566:0x3ffe9dc0 0x400f3375:0x3ffe9de0 0x400f342a:0x3ffe9e00 0x400d7e05:0x3ffe9e40 0x40101edc:0x3ffe9e70 0x40101f59:0x3ffe9ee0 0x400f76b1:0x3ffe9f10 0x400f8b4d:0x3ffe9f40 0x400e1698:0x3ffe9f60 0x400da9dd:0x3ffe9f90 0x400daa0e:0x3ffe9fb0 0x400dc995:0x3ffe9fd0 0x400d9ed3:0x3ffe9ff0 0x400d9b8d:0x3ffea010 0x400d8298:0x3ffea110 0x400d76b2:0x3ffea130 0x40140d5d:0x3ffea150 0x40141046:0x3ffea170 0x40141075:0x3ffea1e0 0x400958d5:0x3ffea200
0x40081e72: panic_abort at /Users/keanlim/tools/esp-idf/components/esp_system/panic.c:402
M5-panic-loop.txt
Bug prevalence
Always
GitHub hash of the SDK that was being used
bba4182d61d56510f628307aff9844b75fb27b0d
Platform
esp32
Platform Version(s)
ESP IDF 3.3.2
Anything else?
No response

@shubhamdp @dhrishi

Hi @kean-apple can you confirm that you are really using IDF v3.3.2?

You should be using IDF v4.4.3.

Updated to ESP-IDF 4.4.3 and using SHA 5400de1cf831c047e2bb1618e2878861fc0186b9 and the crash still happens..
M5-crash-ESP-IDF-4-4-3.txt

I have been seeing this too, SHA ec031378c660fee6650c61c10da3ff0388244e47
I'm on ESP-IDF v4.4.2.


Feature/use zkp packer

PR is OK,  but we need to support 'parser' for old format too. don't we?

PR is OK, but we need to support 'parser' for old format too. don't we?

We don't have old format.


Examples for the RecSys 2021 tutorial on End-2-End Session-based recommendation

[ ] Preproc with NVTabular
[ ] Training with Transformers4Rec
[ ] Inference with Triton

Hi, I'm getting the following error during the example notebook.
dataset = nvt.Dataset(interactions_merged_df)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/tmp/ipykernel_10047/1382503116.py in <module>
----> 1 nvt.Dataset(interactions_merged_df)

/opt/conda/lib/python3.7/site-packages/nvtabular/io/dataset.py in __init__(self, path_or_source, engine, npartitions, part_size, part_mem_fraction, storage_options, dtypes, client, cpu, base_dataset, schema, **kwargs)
    318                     )
    319                 else:
--> 320                     raise ValueError("Only parquet, csv, and avro supported (for now).")
    321             else:
    322                 self.engine = engine(

ValueError: Only parquet, csv, and avro supported (for now)

@Ahanmr hello.  Are you using a container? if yes which docker container? If not, please provide details about your env...
To be able to run the end-to-end notebook, you need two containers:
merlin-pytorch-training:21.12  container to run the entire notebook. First pull the container and launch it. Then, in the container you can git clone the Transformers4rec and then do
cd Transformers4Rec
git checkout main
git pull origin main
pip install -e .

Then  launch the jupyter lab and run the notebooks for ETL and training.
For the inference part of the notebook, you need merlin-inference container only to serve/load the models to Triton inference server. In order to do that please follow the explanations in the notebook:
Launch the docker container
docker run -it --gpus device=0 -p 8000:8000 -p 8001:8001 -p 8002:8002 -v <path_to_saved_models>:/workspace/models/ nvcr.io/nvidia/merlin/merlin-inference:21.12

This script will mount your local model-repository folder that includes your saved models from the previous cell to /workspace/models directory in the merlin-inference docker container.
Start triton server
After you started the merlin-inference container, you can start triton server with the command below. You need to provide correct path of the models folder.
tritonserver --model-repository=<path_to_models> --model-control-mode=explicit
Note: The model-repository path for our example is /workspace/models. The models haven't been loaded, yet. Below, we will request the Triton server to load the saved ensemble model below.

from transformers4rec.torch.utils.examples_utils import fit_and_evaluate
OT_results = fit_and_evaluate(recsys_trainer, start_time_index=7, end_time_index=8, input_dir='./preproc_sessions_by_day')
I'm trying to run this snippet in the notebook but I'm getting this error, I believe that something here isn't accessible to the model.fit(), any idea why this might be happening?
***** Launch training for day 7: *****
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/tmp/ipykernel_25823/251429872.py in <module>
      1 from transformers4rec.torch.utils.examples_utils import fit_and_evaluate
----> 2 OT_results = fit_and_evaluate(recsys_trainer, start_time_index=7, end_time_index=8, input_dir='./preproc_sessions_by_day')

/opt/conda/lib/python3.7/site-packages/transformers4rec/torch/utils/examples_utils.py in fit_and_evaluate(trainer, start_time_index, end_time_index, input_dir)
     75         trainer.train_dataset_or_path = train_paths
     76         trainer.reset_lr_scheduler()
---> 77         trainer.train()
     78
     79         # 3. Evaluate on valid data of time_index+1

/opt/conda/lib/python3.7/site-packages/transformers/trainer.py in train(self, resume_from_checkpoint, trial, ignore_keys_for_eval, **kwargs)
   1113
   1114         # Data loader and number of training steps
-> 1115         train_dataloader = self.get_train_dataloader()
   1116
   1117         # Setting up training control variables:

/opt/conda/lib/python3.7/site-packages/transformers4rec/torch/trainer.py in get_train_dataloader(self)
    148             drop_last=self.args.dataloader_drop_last,
    149             shuffle=True,
--> 150             shuffle_buffer_size=self.args.shuffle_buffer_size,
    151         )
    152

/opt/conda/lib/python3.7/site-packages/transformers4rec/torch/utils/data_utils.py in from_schema(cls, schema, paths_or_dataset, batch_size, max_sequence_length, continuous_features, categorical_features, targets, collate_fn, shuffle, buffer_size, parts_per_chunk, separate_labels, named_labels, sparse_names, sparse_max, **kwargs)
    323                 sparse_max=sparse_max,
    324                 schema=schema,
--> 325                 **kwargs,
    326             )
    327

/opt/conda/lib/python3.7/site-packages/transformers4rec/torch/utils/data_utils.py in __init__(self, paths_or_dataset, batch_size, max_sequence_length, conts, cats, labels, collate_fn, engine, buffer_size, reader_kwargs, shuffle, seed_fn, parts_per_chunk, device, global_size, global_rank, sparse_names, sparse_max, sparse_as_dense, drop_last, schema, **kwargs)
    225             self.drop_last = drop_last
    226
--> 227             self.set_dataset(buffer_size, engine, reader_kwargs)
    228
    229             loader = DataLoader(

/opt/conda/lib/python3.7/site-packages/transformers4rec/torch/utils/data_utils.py in set_dataset(self, buffer_size, engine, reader_kwargs)
    261                 buffer_size,
    262                 engine,
--> 263                 reader_kwargs,
    264             )
    265             self.dataset = dataset

~/Transformers4Rec/examples/end-to-end-session-based/merlin_standard_lib/utils/misc_utils.py in validate_dataset(paths_or_dataset, batch_size, buffer_size, engine, reader_kwargs)
    237     else:
    238         reader_kwargs["part_mem_fraction"] = buffer_size
--> 239     return Dataset(files, engine=engine, **reader_kwargs)

/opt/conda/lib/python3.7/site-packages/nvtabular/io/dataset.py in __init__(self, path_or_source, engine, npartitions, part_size, part_mem_fraction, storage_options, dtypes, client, cpu, base_dataset, schema, **kwargs)
    300                 if engine == "parquet":
    301                     self.engine = ParquetDatasetEngine(
--> 302                         paths, part_size, storage_options=storage_options, cpu=self.cpu, **kwargs
    303                     )
    304                 elif engine == "csv":

/opt/conda/lib/python3.7/site-packages/nvtabular/io/parquet.py in __init__(self, paths, part_size, storage_options, row_groups_per_part, legacy, batch_size, cpu, **kwargs)
    279         if row_groups_per_part is None:
    280             self._real_meta, rg_byte_size_0 = run_on_worker(
--> 281                 _sample_row_group, self._path0, self.fs, cpu=self.cpu, memory_usage=True
    282             )
    283             row_groups_per_part = self.part_size / rg_byte_size_0

/opt/conda/lib/python3.7/site-packages/nvtabular/utils.py in run_on_worker(func, *args, **kwargs)
    178         return dask.delayed(func)(*args, **kwargs).compute()
    179     # No Dask client - Use simple function call
--> 180     return func(*args, **kwargs)

/opt/conda/lib/python3.7/site-packages/nvtabular/io/parquet.py in _sample_row_group(path, fs, cpu, n, memory_usage)
   1170             _df = pq.ParquetFile(f0).read_row_group(0).to_pandas()
   1171     else:
-> 1172         if cudf.utils.ioutils._is_local_filesystem(fs):
   1173             # Allow cudf to open the file if this is a local file
   1174             # system (can be significantly faster in this case)

AttributeError: 'NoneType' object has no attribute 'utils'

Also is there an automated way to generate schema.pb for my dataset?

from transformers4rec.torch.utils.examples_utils import fit_and_evaluate OT_results = fit_and_evaluate(recsys_trainer, start_time_index=7, end_time_index=8, input_dir='./preproc_sessions_by_day')
I'm trying to run this snippet in the notebook but I'm getting this error, I believe that something here isn't accessible to the model.fit(), any idea why this might be happening?
***** Launch training for day 7: *****
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/tmp/ipykernel_25823/251429872.py in <module>
      1 from transformers4rec.torch.utils.examples_utils import fit_and_evaluate
----> 2 OT_results = fit_and_evaluate(recsys_trainer, start_time_index=7, end_time_index=8, input_dir='./preproc_sessions_by_day')

/opt/conda/lib/python3.7/site-packages/transformers4rec/torch/utils/examples_utils.py in fit_and_evaluate(trainer, start_time_index, end_time_index, input_dir)
     75         trainer.train_dataset_or_path = train_paths
     76         trainer.reset_lr_scheduler()
---> 77         trainer.train()
     78
     79         # 3. Evaluate on valid data of time_index+1

/opt/conda/lib/python3.7/site-packages/transformers/trainer.py in train(self, resume_from_checkpoint, trial, ignore_keys_for_eval, **kwargs)
   1113
   1114         # Data loader and number of training steps
-> 1115         train_dataloader = self.get_train_dataloader()
   1116
   1117         # Setting up training control variables:

/opt/conda/lib/python3.7/site-packages/transformers4rec/torch/trainer.py in get_train_dataloader(self)
    148             drop_last=self.args.dataloader_drop_last,
    149             shuffle=True,
--> 150             shuffle_buffer_size=self.args.shuffle_buffer_size,
    151         )
    152

/opt/conda/lib/python3.7/site-packages/transformers4rec/torch/utils/data_utils.py in from_schema(cls, schema, paths_or_dataset, batch_size, max_sequence_length, continuous_features, categorical_features, targets, collate_fn, shuffle, buffer_size, parts_per_chunk, separate_labels, named_labels, sparse_names, sparse_max, **kwargs)
    323                 sparse_max=sparse_max,
    324                 schema=schema,
--> 325                 **kwargs,
    326             )
    327

/opt/conda/lib/python3.7/site-packages/transformers4rec/torch/utils/data_utils.py in __init__(self, paths_or_dataset, batch_size, max_sequence_length, conts, cats, labels, collate_fn, engine, buffer_size, reader_kwargs, shuffle, seed_fn, parts_per_chunk, device, global_size, global_rank, sparse_names, sparse_max, sparse_as_dense, drop_last, schema, **kwargs)
    225             self.drop_last = drop_last
    226
--> 227             self.set_dataset(buffer_size, engine, reader_kwargs)
    228
    229             loader = DataLoader(

/opt/conda/lib/python3.7/site-packages/transformers4rec/torch/utils/data_utils.py in set_dataset(self, buffer_size, engine, reader_kwargs)
    261                 buffer_size,
    262                 engine,
--> 263                 reader_kwargs,
    264             )
    265             self.dataset = dataset

~/Transformers4Rec/examples/end-to-end-session-based/merlin_standard_lib/utils/misc_utils.py in validate_dataset(paths_or_dataset, batch_size, buffer_size, engine, reader_kwargs)
    237     else:
    238         reader_kwargs["part_mem_fraction"] = buffer_size
--> 239     return Dataset(files, engine=engine, **reader_kwargs)

/opt/conda/lib/python3.7/site-packages/nvtabular/io/dataset.py in __init__(self, path_or_source, engine, npartitions, part_size, part_mem_fraction, storage_options, dtypes, client, cpu, base_dataset, schema, **kwargs)
    300                 if engine == "parquet":
    301                     self.engine = ParquetDatasetEngine(
--> 302                         paths, part_size, storage_options=storage_options, cpu=self.cpu, **kwargs
    303                     )
    304                 elif engine == "csv":

/opt/conda/lib/python3.7/site-packages/nvtabular/io/parquet.py in __init__(self, paths, part_size, storage_options, row_groups_per_part, legacy, batch_size, cpu, **kwargs)
    279         if row_groups_per_part is None:
    280             self._real_meta, rg_byte_size_0 = run_on_worker(
--> 281                 _sample_row_group, self._path0, self.fs, cpu=self.cpu, memory_usage=True
    282             )
    283             row_groups_per_part = self.part_size / rg_byte_size_0

/opt/conda/lib/python3.7/site-packages/nvtabular/utils.py in run_on_worker(func, *args, **kwargs)
    178         return dask.delayed(func)(*args, **kwargs).compute()
    179     # No Dask client - Use simple function call
--> 180     return func(*args, **kwargs)

/opt/conda/lib/python3.7/site-packages/nvtabular/io/parquet.py in _sample_row_group(path, fs, cpu, n, memory_usage)
   1170             _df = pq.ParquetFile(f0).read_row_group(0).to_pandas()
   1171     else:
-> 1172         if cudf.utils.ioutils._is_local_filesystem(fs):
   1173             # Allow cudf to open the file if this is a local file
   1174             # system (can be significantly faster in this case)

AttributeError: 'NoneType' object has no attribute 'utils'

@Ahanmr I will look into that. Did you check if your data is in the input_dir='./preproc_sessions_by_day' path? Do you see folders for each day under the /preproc_sessions_by_day' and parquet files in each one?  If not that means you did not preprocess the dataset properly? Which notebook are you running btw?
Also please open another issue because I am going to close this one, since this task is completed.

Also is there an automated way to generate schema.pb for my dataset?

Yes with NVTabular you can do it.

Closing since this task was completed.


Revert back the test of deployment

Coverage remained the same at 31.394% when pulling a267102626c379ca8e2c6d060562b3b3317c41e7 on stvyang-m:dev into 220c11a58a4ebb5d5a479fa56717139d7c27dd2c on danliris:dev.


compatibility update, port data load code to tf.data, etc.

minor changes to make this repo be compatible with tf 1.10.

port data load code to tf.data, https://github.com/tensorflow/tensorflow/blob/r1.4/tensorflow/contrib/data/README.md
remove warning, "softmax_cross_entropy_with_logits deprecated", https://github.com/tensorflow/minigo/issues/37

For anyone not sure how to make the changes.

Use data = tf.data.Dataset instead of tf.contrib.data. Same goes to Iterator
Use  tf.nn.softmax_cross_entropy_with_logits_v2 instead of  tf.nn.softmax_cross_entropy_with_logits

I'll look over this during the next week. Have some trouble with our newborn at the moment and I don't find much time to cross check the PR. But will come back to this, I promise


Fix: contrib/journal: `buffer-file-name' is nil in Capture buffer

Fixes #2923

Good catch. Thanks!


evaluate.py is not updated to sdk

Rasa Core version: master
Python version:
Operating system (windows, osx, ...):
Issue:
current evaluation script doesn't work
Content of domain file (if used & relevant):

Will be fixed in this PR: https://github.com/RasaHQ/rasa_core/pull/550


Request for comment: Proposed APIs to give users control over what websites projects can access

The problem
A project with eg. the fetch extension can talk to any server it wants to without confirmation. This can have privacy and security implications that are not obvious. Should the fetch extension really allow port scanning and IP logging by default? Probably not.
Proposed solution
Part 1 is to replace every use of fetch, WebSocket, Image, etc. with eg. Scratch.fetch, Scratch.WebSocket, Scratch.Image, etc. Initially these could just be copies of their window.* counterparts. We can probably configure ESLint to error if using the old APIs.
Part 2 is to make those APIs check if the origin they are trying to load is allowed or not. We might present a prompt like this:

This project is trying to access the website "trampoline.turbowarp.org".
Allowing this may share your IP address and approximate location with the website.
[Deny]
[Allow once]
[Always allow trampoline.turbowarp.org]
[Temporarily allow all websites]

Sort of like our own "google.com wants to know your location [Deny] [Allow]" prompt
Until the user chooses, fetch()'s promise will remain unresolved, WebSocket won't emit open, etc. If user denies, an error will be returned in the same way that the native fetch() would return a rejected promise, WebSocket would emit an error event, etc.
Having an option to allow forever is safe here as this is just fetching; we're not executing arbitrary JS
Depending on how fetch() handles HTTP redirects, we can likely add a builtin exception for trusted static site hosts like *.github.io where there is no privacy concern
We can do a similar thing for window opening and page redirecting:

This project is trying to open a new tab to "github.com"
[... the same options ...]

For these we can also add checks to ensure URLs like javascript:alert(1) do not run code in just one spot instead of everywhere. We can also probably detect popup blockers.
Extensions running in a sandbox will be exempted from these checks (there is nothing we can do to force them to use Scratch.fetch instead of window.fetch). The prompt to confirm loading them will be updated to mention this.
The packager will always be exempted from these checks.
Concerns

These confirmations will be very annoying very quickly
Notification fatigue

The only problem i have with this is that users could break projects, because they have required data on another server...
This also breaks CL. As its entire use is connecting to a Cloudlink websocket server

It doesn't "break" cloud link; the user just has to consent to connecting. Perhaps one of the default servers can be added to the list of always allowed websites.
A user denying permission will behave the same as when someone doesn't have an internet connection. Projects should already be designed with that condition in mind.

I don't think the notifications will be a problem, especially if the projects are designed with needing to ask for permission in mind.


How to modify timeouts when script is waiting on webpage

How to modify timeout values when waiting for a response on a webpage?
I'm interacting with a web page that waits for ~2-3min prior to responding to the client.
and In this case, that is expected.
So far I am managing this by wrapping into try block, but this way I'm skipping on other errors as well.
$Driver = Start-SeChrome
Enter-SeUrl https://www.example.com -Driver $Driver
$Element=Find-SeElement -Driver $Driver -Wait -Timeout 10 -name "Button-of-slowness"
try
{
Invoke-SeClick -Element $Element
# executes for 2-3 mins
} catch {
#do nothing
}

Hey, I see this is open for a while.
I have took the maintainer role of this repository and
I plan to have V4 released on the first week of October.
Amongst the change, the addition of Wait-SeDriver / Wait-SeElement should allow you to achieve what you wish to do simply enough by allowing you to wait for specific conditions before continuing.
You can see #125 for more details.
V4

Start-SeDriver -Browser Chrome
Set-SeUrl -Url 'https://www.example.com'
$Element = Get-SeElement -By Name -Value "Button-of-slowness"

Send-SeClick -Element $Element

# If the button get removed from the DOM when the response is received (EG: navigated to another page)
Wait-SeElement -Element $Element -Condition StalenessOf -Timeout 500

# If you want to wait for a specific component
Wait-SeElement -By Name -Value 'SomeOtherComponenentLoaded' -Timeout 500

For the current version, you could wrap something like this

Function Wait-SeElementExists($Driver,$Element,$Timeout) {
    $WebDriverWait = [OpenQA.Selenium.Support.UI.WebDriverWait]::new($Driver, (New-TimeSpan -Seconds $Timeout))
    $SeCondition = [OpenQA.Selenium.Support.UI.ExpectedConditions]::ElementExists($Element)
    try {
        [void]($WebDriverWait.Until($SeCondition))
        return $true
    }
    catch {
        Write-Error $_
        return $false
    }
}

(You can take a look at the other conditions that can be used here: https://www.selenium.dev/selenium/docs/api/dotnet/html/Methods_T_OpenQA_Selenium_Support_UI_ExpectedConditions.htm)


(HOLD) JS Prototype with isolated components (HTML. CSS, JS, data layer)

TODO

[ ]

Outcomes

Generate discussion with the product pod
Identify gaps and questions to discuss now, later

Deliverables

presentation at review

Estimated Effort

[ ] XS: 1 to 2 hours
[ ] S: half a day
[ ] M: 1-2 day
[ ] L: 1 week
[ ] XL: Move to an epic

Out of scope

discussion to happen before jumping into it during product pod

Started from the existing work completed in wet-boew-experimental.
The following was done:

Integrated community contribution + made adjustment as required (Jekyll template setup + CSS variable set)
Removed requireJS in favor of ES Module (import && export)
Added a Fetch option that allow load securely third party javascript in the page.
Updated all the working example to ensure they continue to work as expected

Next:

Reproduce the IA structure (component based) similar to we the structure we experimented/tried with GCWeb.
Create a working example

Here an example where we are polyfilling CSS stage 0 in browser
https://wet-boew.github.io/wet-boew-experimental/src/sites/wb.dev-css/example.html


Telemetry: propogate hostname where possible

Fixes https://github.com/istio/istio/issues/39196, I think
Please provide a description of this PR:

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

as https://github.com/istio/istio/issues/39196#issuecomment-1157067351 mentioned, replaced by https://github.com/istio/istio/pull/39476/files


WebDriverAgent runner installs in a repeated loop on iOS 17.3

Hi, I have followed the instructions given in GADS device provider & GADS UI. When I try to launch the UI, i see that the WebDriverAgent runner is getting installed repeatedly in a loop on the device. When I look at the logs, it shows an error:
"Error waiting for WebDriverAgent(xcodebuild) command to finish.. it is errored out or device <> disconnected agent killed"
I am not sure what is wrong with the setup. Can you advise?
Env -> iOS 17.3 and mac os ventura 13.6.4

Lol, I've never seen agent killed error before :sweat_smile: Ventura means you are not using Xcode 15? I would assume this is the reason for iOS 17 failures on first glance. Have you set up the signing for WebDriverAgent? Did you try running it on a device through Xcode directly?

I am using  Xcode build15.2  and signing profile and other things have taken care.  Here is something I have observed

when I run xcodebuild test-without-building command via terminal . I see that webdriver agent runner is working as expected and http end point deviceip/status returns response
But when I run xcodebuild test-with-building command installs webdriveragentrunner repeatedly in a loop.
I tried to modify one of the files iOS.go , line 76 . It did not help.
I assume webdriveragent.ipa file is not needed for ios.
Can you advise?

The build-for-testing command is executed only once on provider startup, this is to speed up deployment on each device later with test-without-building. In general at the moment *.ipa file is not needed because the provider uses xcodebuild directly to deploy on the devices from the already built WebDriverAgent code. Do you have paid developer account? Also I was asking if you tried running WebDriverAgent from Xcode directly, maybe the developer needs to be trusted from the device settings.

I would suggest you try out the develop branches for both the provider and the UI. There are a few differences in setup and you'll have to drop the current MongoDB data, but in theory it should work better

I have got a Developer account. It worked when I ran directly with Xcode . The build was successful, trusted the developer on the device.. I can see the <deviceip>/status endpoint returns valid response.
Let me try with develop branch.
Thank you!

I've created a Discord server that is in the develop branch readme, you can join if you want so we can chat/debug faster :P


Removing .git from dockerignore

It looks like the command backstage-cli app:build requires the git metadata.  With .git in the dockerignore file, I saw this error:
➜  backstage git:(master) docker build -t backstage-frontend -f ./contrib/docker/multi-stage-frontend/Dockerfile .

Sending build context to Docker daemon  87.31MB
Step 1/13 : FROM node:12-buster AS build
 ---> b247258c051b
Step 2/13 : RUN mkdir /app
 ---> Using cache
 ---> 19921b2b70f3
Step 3/13 : COPY . /app
 ---> Using cache
 ---> adf3190957a7
Step 4/13 : WORKDIR /app
 ---> Using cache
 ---> d57d193dd053
Step 5/13 : RUN yarn install
 ---> Using cache
 ---> 23cbb5d4cdbd
Step 6/13 : RUN yarn workspace example-app build
 ---> Running in f1de25e8396d
yarn workspace v1.22.1
yarn run v1.22.1
$ backstage-cli app:build
Loaded config from app-config.yaml
fatal: not a git repository (or any of the parent directories): .git
WARNING: Failed to read git commit, ExitCodeError: Command 'git rev-parse HEAD' exited with code 128
fatal: not a git repository (or any of the parent directories): .git
WARNING: Failed to describe git version, ExitCodeError: Command 'git describe --always' exited with code 128

Hey, I just made a Pull Request!

:heavy_check_mark: Checklist

[ ] A changeset describing the change and affected packages. (more info)
[ ] Added or updated documentation
[ ] Tests for new functionality and regression tests for bug fixes
[ ] Screenshots attached (for UI changes)

We recommend building the packages on the host in CI rather than within dockerfile, so the default setup is optimized for that

Thanks for the info @Rugvip


main task pointer (IDFGH-9175)

I want to apply vTaskSuspend and vTaskReume to MainTask. How can I find the main task pointer?

You can call xTaskGetCurrentTaskHandle() from app_main() and store the resulting handle somewhere for later use.

Thanks for reporting, will close due to short of feedback, feel free to reopen with more updates. Thanks.


Is this wrong version code?

Is this wrong version code  or does it changing for different countries?
Because when I check on Instagram's original App User-Agent is;
Instagram <IP_ADDRESS>.123 Android (27/8.1.0; 440dpi; 1080x2030; Xiaomi/xiaomi; Redmi Note 5; whyred; qcom; en_US; 180322800)

@Nerixyz need your clarification! :)

No, its right


Handle fcntl()'s

We already handle a few, but not all. Marking as release blocking.

We're handling the easy ones.


[TASK] v1.1.3 dependencies

Bump v1.1.3 dependencies

Deps

[x] Longhorn (https://github.com/harvester/harvester/pull/4672)
Kubevirt

[x] https://github.com/harvester/harvester/pull/4598#top
[x] newer kubevirt https://github.com/harvester/harvester/pull/4718

[x] harvester-network-controller (v0.3.5)
[x] harvester-node-disk-manager (v0.5.2)
[x] kube-vip (waiting for @w13915984028 to confirm, Jian suggested not bump)
[x] Deprecate kube-vip-cloud-provider? (@w13915984028 suggest not bump, too many changes)
[x] harvester-load-balancer (v0.1.6)
[x] whereabouts (https://github.com/harvester/harvester/pull/4261)
[x] harvester-node-manager (v0.1.8)
[x] support-bundle-kit (PR in review) (v0.0.32)
[x] Update values.yaml file

[x] Add-ons (Need a new tag, new chart, and bump add-on versions in both harvester/harvester and harvester/installer)

[x] pcidevice
[x] vm-import-controller

[x] Check monitoring and logging versions (not addon yet), Jian suggest not bump.
[x] RKE2 (v1.25.15, v1.24.17) https://github.com/harvester/harvester-installer/pull/592
[x] Rancher (v2.6.13) https://github.com/harvester/harvester-installer/pull/592
[x] OS https://github.com/harvester/harvester-installer/pull/593
[x] Go versions

[x] harvester/harvester https://github.com/harvester/harvester/pull/4732
[x] harvester/installer. https://github.com/harvester/harvester-installer/pull/590

Provide test result on a local 3 nodes upgrade.
Environment

Harvester

Version: v1.1.1 -> v1.1.3-rc1
Profile: QEMU/KVM, 3 nodes (8C/16G/500G)
ui-source: Auto

Checks

Upgrade cluster from v1.1.1 -> v1.1.3-rc1

Check dependencies version

Deps
Description[^1]
valus.yaml[^2]
Test Env.
Comment

Longhorn

kubevirt

vmdp

vmdp

vmdp

vmdp

vmdp

vmdp

vmdp

vmdp

vmdp

vmdp

vmdp

[^1]: Description of this ticket
[^2]: https://github.com/harvester/harvester/blob/v1.1.3-rc1/deploy/charts/harvester/values.yaml
Deps
Longhorn (https://github.com/harvester/harvester/pull/4672)
Kubevirt
https://github.com/harvester/harvester/pull/4598#top
newer kubevirt https://github.com/harvester/harvester/pull/4718
harvester-network-controller (v0.3.5)
harvester-node-disk-manager (v0.5.2)
kube-vip (waiting for @w13915984028 to confirm, Jian suggested not bump)
Deprecate kube-vip-cloud-provider? (@w13915984028 suggest not bump, too many changes)
harvester-load-balancer (v0.1.6)
whereabouts (https://github.com/harvester/harvester/pull/4261)
harvester-node-manager (v0.1.8)
support-bundle-kit (https://github.com/rancher/support-bundle-kit/pull/85 in review) (v0.0.32)
Update values.yaml file
Add-ons (Need a new tag, new chart, and bump add-on versions in both harvester/harvester and harvester/installer)
pcidevice
vm-import-controller
Check monitoring and logging versions (not addon yet), Jian suggest not bump.
RKE2 (v1.25.15, v1.24.17) https://github.com/harvester/harvester-installer/pull/592
Rancher (v2.6.13) https://github.com/harvester/harvester-installer/pull/592
OS https://github.com/harvester/harvester-installer/pull/593
Go versions
harvester/harvester https://github.com/harvester/harvester/pull/4732
harvester/installer. https://github.com/harvester/harvester-installer/pull/590

(WIP)
Provide test result on a local 3 nodes upgrade.
Environment

Harvester

Version: v1.1.1 -> v1.1.3-rc1
Profile: QEMU/KVM, 3 nodes (8C/16G/500G)
ui-source: Auto

Checks

Upgrade cluster from v1.1.1 -> v1.1.3-rc1

Check dependencies version

Deps
Description[^1]
Code
Test Env.
Comment

Longhorn
1.3.3
1.3.3 [^5]

kubevirt
0.54.0-1504<IP_ADDRESS>
0.54.0-1504<IP_ADDRESS> [^2]

vmdp
<IP_ADDRESS>
<IP_ADDRESS> [^10]

harvester-network-controller
0.3.5
0.3.5 [^2]

harvester-node-disk-manager
0.5.2
0.5.2 [^2]

kube-vip
not bump
0.5.10 [^2]

kube-vip-cloud-provider
not bump
0.0.1 [^2]

harvester-load-balancer
0.1.6
0.1.6 [^2]

whereabouts
0.6.2
0.6.2-amd64 [^2]

harvester-node-manager
0.1.8
0.1.8 [^2]

support-bundle-kit
0.0.32
0.0.32 [^2]

addon: pcidevice
?
0.2.6 [^6]

addon: vm-import-controller
?
0.1.6 [^6]

addon? monitoring
not bump
100.1.0+up19.0.3 [^9]

addon? logging
not bump
100.1.3+up3.17.7 [^8]

RKE2
1.24.17
v1.24.17+rke2r1 [^11]

Rancher
2.6.13
2.6.13 [^3]

OS
1.1-20231115
1.1-20231121 [^12]

Go
1.20.10
1.20.10-bookworm [^4]

[^1]: Description of this ticket
[^2]: https://github.com/harvester/harvester/blob/v1.1.3-rc1/deploy/charts/harvester/values.yaml
[^3]: https://github.com/harvester/harvester-installer/blob/v1.1.3-rc1/scripts/version-rancher
[^4]: https://github.com/harvester/harvester/blob/v1.1.3-rc1/Dockerfile.dapper
[^5]: https://github.com/harvester/harvester/blob/v1.1.3-rc1/deploy/charts/harvester/Chart.yaml
[^6]: https://github.com/harvester/harvester/tree/v1.1.3-rc1/package/upgrade/addons
[^7]: https://github.com/harvester/harvester-installer/blob/v1.1.3-rc1/scripts/package-harvester-os
[^8]: https://github.com/harvester/harvester-installer/blob/v1.1.3-rc1/scripts/version-logging
[^9]: https://github.com/harvester/harvester-installer/blob/v1.1.3-rc1/scripts/version-monitoring
[^10]: https://github.com/harvester/harvester/blob/v1.1.3-rc1/pkg/data/template.go
[^11]: https://github.com/harvester/harvester-installer/blob/v1.1.3-rc1/scripts/version-rke2
[^12]: https://github.com/harvester/harvester-installer/blob/v1.1.3-rc1/scripts/package-harvester-os


add error names to ensure error names work with webpack + uglify

Set name for errors to allow for the field to be used in mangled code. Addresses #600

Hi @ledbit, I will discuss this change with the team. Where are you using KafkaJS with webpack + uglifier? Because, you might find other challenges.

We're bundling it as part of our single file, NodeJS (10.x) application. What kind of other issues/challenges are you thinking we'd run into?

I think he was wondering if you were trying to run it in a non-Node environment (such as browsers, which won't work out of the box), since it's somewhat unusual to do bundling and minification in Node.

Thanks for the contribution! This will be available in 1.12.0-beta.14 as soon as this build is done.


Some possible improvements and new APIs

maxHeight works in auto-layout system. Added numberOfLines api.
Added lineSpace api.
Added normalBackgroundColor api.
WSTagsView repects layoutMargins.
Added tagLayoutMargins api.
Reposition tag views on bounds change.
Unavailable padding, use contentInset instead.

@billzhou0223 Thanks, I'll review this later today.


feat: Add methods we'll need to build a django integration

The Pymemcache package allows us to inject a hasher which needs to implement the protocol of get_node, add_node, remove_node. Adding those methods to Ketama allows us to build a very simple wrapper around the Rust binding that will work nicely with Django
Paired with @anonrig on this.

Closing in favour of #54


fix: add review doesn't behave as expected

I figured out that the review note creation doesn't behave as I expected then I made this fix to make it work with redux correctly

( old way ) Here, I make a request to have the list of all the notes since Patrick did not return me the created note that's why I asked him to do it. You can see a loader due to loading state change
https://user-images.githubusercontent.com/54010836/142156986-b03c1bfb-79d1-47c1-be69-f79e410d7ed3.mov
( the fix ) Here, instead of making a query to get the notes list I push the note returned after creation in the reducer which avoids a reloading of the note list. We no longer have a loading state change
https://user-images.githubusercontent.com/54010836/142157153-24f4cbdc-4b68-4917-86de-390c247235b7.mov

@elysee15 I've tested the note functionality as follows:
1. Went to a page with no note and checked that it said "No Reviewer Notes"

2. Created a note, left the page and went back to make sure there was no "No Reviewer Notes" text and the note was displayed.

3. Updated the note, navigated to the dashboard and back again.

4. Added another note, updated the first note.

Everything works as expected; however I'm concerned that these changes seem to indicate a workflow that was in error that I didn't try above. Could you indicate how I could create the problem that you're fixing?
Would it be possible to add a story for review notes tests to make sure that things work as expected?
Finally would you mind setting the text area for the review note rows=3 instead of 7 and sending me a screenshot? The review notes text area seems a little large right now and dominates the space, I'd be interested to see what it looks like in a bit more compact of a form.

( old way ) Here, I make a request to have the list of all the notes since Patrick did not return me the created note that's why I asked him to do it. You can see a loader due to loading state change https://user-images.githubusercontent.com/54010836/142156986-b03c1bfb-79d1-47c1-be69-f79e410d7ed3.mov
( the fix ) Here, instead of making a query to get the notes list I push the note returned after creation in the reducer which avoids a reloading of the note list. We no longer have a loading state change https://user-images.githubusercontent.com/54010836/142157153-24f4cbdc-4b68-4917-86de-390c247235b7.mov

Excellent, thank you for the explanation! I tested locally and I don't see the loading message.

@elysee15 your change looks very good and I like it a lot, but I'm not seeing it in this PR:

I think we might be having rebasing issues that's causing trouble. Same with #176

Also would  you please change the text in the deletion confirm alert to:
"Are you sure you want to delete this reviewer's note?"


[DO NOT MERGE]See Perf Impact of Deep Buffering

API change check
API changes are not detected in this pull request.

Closing as there wasn't too much of a performance hit and there are ways to mitigate the performance hit


Basa in apollo_contrib no longer works with Ubuntu 18

The basa kernel is not building / loading with Ubuntu 18.
Open issues related to this:
https://github.com/ApolloAuto/apollo-contrib/issues/22
https://github.com/ApolloAuto/apollo-contrib/issues/21
Is Ubuntu 18 currently supported?
Thanks!

more info about the system, Ubuntu, kernel and gcc?
Ubuntu 18.04.2 LTS+4.15.0-29-generic+gcc 7.4 have no problem from the tip

Updating basa driver for 5.x kernels is done in PR ApolloAuto/apollo-contrib#29 . The issue here will be closed.


Removed all_frames line from manifest.json: defaults to false

Fixes #2
This will stop multiple instances of Codext being triggered in the background

Thanks for the investigation and the fix!